{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用Char RNN对姓名进行分类\n",
    "我们下面会训练一个字符级别的RNN模型来预测一个姓名是哪个国家人的姓名。我们的数据集收集了18个国家的近千个人名(英文名，注意中国人也是英文名，否则就是语言识别问题了），我们最终的模型就可以预测这个姓名是哪个国家的人：\n",
    "\n",
    "$ python predict.py Hinton\n",
    "(-0.47) Scottish\n",
    "(-1.52) English\n",
    "(-3.57) Irish\n",
    "\n",
    "$ python predict.py Schmidhuber\n",
    "(-0.19) German\n",
    "(-2.48) Czech\n",
    "(-2.68) Dutch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备数据\n",
    "在data/names目录下有18个文本文件，命名规范为[语言].txt。每个文件的每一行都是一个人名。此外，我们实现了一个unicode_to_ascii把诸如à之类转换成a。\n",
    "\n",
    "最终我们得到一个字典category_lines， {language: [names ...]}。key是语言名，value是名字的列表。all_letters里保存所有的字符。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['../data/names/Portuguese.txt', '../data/names/Korean.txt', '../data/names/Greek.txt', '../data/names/English.txt', '../data/names/Italian.txt', '../data/names/Dutch.txt', '../data/names/Irish.txt', '../data/names/Spanish.txt', '../data/names/Vietnamese.txt', '../data/names/Chinese.txt', '../data/names/Arabic.txt', '../data/names/German.txt', '../data/names/Japanese.txt', '../data/names/Scottish.txt', '../data/names/French.txt', '../data/names/Russian.txt', '../data/names/Czech.txt', '../data/names/Polish.txt']\n"
     ]
    }
   ],
   "source": [
    "import glob\n",
    "\n",
    "all_filenames = glob.glob('../data/names/*.txt')\n",
    "print(all_filenames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Slusarski\n"
     ]
    }
   ],
   "source": [
    "import unicodedata\n",
    "import string\n",
    "\n",
    "all_letters = string.ascii_letters + \" .,;'\"\n",
    "n_letters = len(all_letters)\n",
    "\n",
    "# http://stackoverflow.com/a/518232/2809427\n",
    "def unicode_to_ascii(s):\n",
    "    return ''.join(\n",
    "        c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn'\n",
    "        and c in all_letters\n",
    "    )\n",
    "\n",
    "print(unicode_to_ascii('Ślusàrski'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_categories = 18\n"
     ]
    }
   ],
   "source": [
    "category_lines = {}\n",
    "all_categories = []\n",
    "\n",
    "def readLines(filename):\n",
    "    lines = open(filename).read().strip().split('\\n')\n",
    "    return [unicode_to_ascii(line) for line in lines]\n",
    "\n",
    "for filename in all_filenames:\n",
    "    category = filename.split('/')[-1].split('.')[0]\n",
    "    all_categories.append(category)\n",
    "    lines = readLines(filename)\n",
    "    category_lines[category] = lines\n",
    "\n",
    "n_categories = len(all_categories)\n",
    "print('n_categories =', n_categories)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Abandonato', 'Abatangelo', 'Abatantuono', 'Abate', 'Abategiovanni']\n"
     ]
    }
   ],
   "source": [
    "print(category_lines['Italian'][:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 把姓名(String)变成Tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们已经把数据处理好了，接下来需要把姓名从字符串变成Tensor，因为机器学习只能处理数字。\n",
    "\n",
    "为了表示一个字母，我们使用“one-hot\"的表示方法。这是一个长度为<1 x n_letters>的向量，对应字符的下标为1，其余为0。\n",
    "\n",
    "对于一个姓名，我们用大小为<line_length x 1 x n_letters>的Tensor来表示。第二维表示batch大小，因为PyTorch的RNN要求输入是< time x batch x input_features>。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "# 把一个字母变成<1 x n_letters> Tensor\n",
    "def letter_to_tensor(letter):\n",
    "    tensor = torch.zeros(1, n_letters)\n",
    "    letter_index = all_letters.find(letter)\n",
    "    tensor[0][letter_index] = 1\n",
    "    return tensor\n",
    "\n",
    "#  把一行(名字)转换成<line_length x 1 x n_letters>的Tensor\n",
    "def line_to_tensor(line):\n",
    "    tensor = torch.zeros(len(line), 1, n_letters)\n",
    "    for li, letter in enumerate(line):\n",
    "        letter_index = all_letters.find(letter)\n",
    "        tensor[li][0][letter_index] = 1\n",
    "    return tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,\n",
      "          0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,\n",
      "          0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,\n",
      "          0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,\n",
      "          0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]])\n"
     ]
    }
   ],
   "source": [
    "print(letter_to_tensor('A'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 创建网络\n",
    "如果想“手动”创建，那么在PyTorch里创建RNN和全连接网络的代码并没有太大差别。因为PyTorch的计算图是动态实时编译的，不同time-step的for循环不需要“内嵌”在RNN里。因此每个训练数据即使长度不同也没有关系，因为每次都是根据当前的数据长度“实时”编译出来的计算图。\n",
    "网络结构如下图所示：\n",
    "\n",
    "![](network.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "from torch.autograd import Variable\n",
    "\n",
    "class RNN(nn.Module):\n",
    "    def __init__(self, input_size, hidden_size, output_size):\n",
    "        super(RNN, self).__init__()\n",
    "        \n",
    "        self.input_size = input_size\n",
    "        self.hidden_size = hidden_size\n",
    "        self.output_size = output_size\n",
    "        \n",
    "        self.i2h = nn.Linear(input_size + hidden_size, hidden_size)\n",
    "        self.i2o = nn.Linear(input_size + hidden_size, output_size)\n",
    "        self.softmax = nn.LogSoftmax(dim=1)\n",
    "    \n",
    "    def forward(self, input, hidden):\n",
    "        combined = torch.cat((input, hidden), 1)\n",
    "        hidden = self.i2h(combined)\n",
    "        output = self.i2o(combined)\n",
    "        output = self.softmax(output)\n",
    "        return output, hidden\n",
    "\n",
    "    def init_hidden(self):\n",
    "        return Variable(torch.zeros(1, self.hidden_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 手动测试网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_hidden = 128\n",
    "rnn = RNN(n_letters, n_hidden, n_categories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了运行网络的一个step，我们需要传入当前时刻的输入(在这里是一个字母)和前一个隐状态。运行一次之后我们可以得到输出和下一个隐状态。\n",
    "\n",
    "注意PyTorch模块需要变量而不是Tensor。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "output.size = torch.Size([1, 18])\n"
     ]
    }
   ],
   "source": [
    "input = Variable(letter_to_tensor('A'))\n",
    "hidden = rnn.init_hidden()\n",
    "\n",
    "output, next_hidden = rnn(input, hidden)\n",
    "print('output.size =', output.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了高效，我们不会每一个step都创建一个Tensor，因此我们使用line_to_tensor并且使用切片来选择一个字符。更进一步的优化可以使用batch的Tensor。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-2.8509, -2.8969, -2.8901, -2.8594, -2.8999, -2.8880, -2.8469,\n",
      "         -2.9089, -2.8506, -2.9297, -2.9212, -2.8451, -2.9812, -2.9390,\n",
      "         -2.8964, -2.8685, -2.8348, -2.9327]])\n"
     ]
    }
   ],
   "source": [
    "input = Variable(line_to_tensor('Albert'))\n",
    "hidden = Variable(torch.zeros(1, n_hidden))\n",
    "\n",
    "output, next_hidden = rnn(input[0], hidden)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我可以看出，输出是一个<1 x n_categories> 的Tensor，表示输出每个category的概率。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备训练\n",
    "\n",
    "训练之前，我们需要一些工具函数。第一个就是根据网络的输出把它变成分类，我们这里使用Tensor.topk来选取概率最大的那个下标，然后得到分类名称。："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Czech', 16)\n"
     ]
    }
   ],
   "source": [
    "def category_from_output(output):\n",
    "    top_n, top_i = output.data.topk(1) # Tensor out of Variable with .data\n",
    "    category_i = top_i[0].item()\n",
    "    return all_categories[category_i], category_i\n",
    "\n",
    "print(category_from_output(output))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也需要一个函数来随机挑选一个训练数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "category = Italian / line = Pesaresi\n",
      "category = Dutch / line = Nifterik\n",
      "category = Polish / line = Kozlow\n",
      "category = Arabic / line = Atiyeh\n",
      "category = English / line = Thatcher\n",
      "category = Korean / line = Woo\n",
      "category = Czech / line = Prachar\n",
      "category = Greek / line = Close\n",
      "category = Japanese / line = Inouye\n",
      "category = German / line = Cline\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "def random_training_pair():                                                                                                               \n",
    "    category = random.choice(all_categories)\n",
    "    line = random.choice(category_lines[category])\n",
    "    category_tensor = Variable(torch.LongTensor([all_categories.index(category)]))\n",
    "    line_tensor = Variable(line_to_tensor(line))\n",
    "    return category, line, category_tensor, line_tensor\n",
    "\n",
    "for i in range(10):\n",
    "    category, line, category_tensor, line_tensor = random_training_pair()\n",
    "    print('category =', category, '/ line =', line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练网络\n",
    "\n",
    "现在我们可以训练网络了，因为RNN的输出已经取过log了，所以计算交叉熵只需要选择正确的分类对于的值就可以了，PyTorch提高了nn.NLLLoss()函数来实现这个目的，它基本就是实现了loss(x, class) = -x[class]。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = nn.NLLLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们用optimizer而不是自己手动来更新参数，这里我们使用最原始的SGD算法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.005 \n",
    "optimizer = torch.optim.SGD(rnn.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "训练的每个循环如下：Each loop of training will:\n",
    "    创建输入和输出Tensor\n",
    "    创建初始化为零的隐状态Tensor\n",
    "    for each letter in 输入Tensor:\n",
    "        output, hidden=rnn(input,hidden)\n",
    "    计算loss\n",
    "    backward计算梯度\n",
    "    optimizer.step\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(category_tensor, line_tensor):\n",
    "    rnn.zero_grad()\n",
    "    hidden = rnn.init_hidden()\n",
    "    \n",
    "    for i in range(line_tensor.size()[0]):\n",
    "        output, hidden = rnn(line_tensor[i], hidden)\n",
    "\n",
    "    loss = criterion(output, category_tensor)\n",
    "    loss.backward()\n",
    "\n",
    "    optimizer.step()\n",
    "\n",
    "    return output, loss.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们就要用训练数据来训练了。因为上面的函数同时返回输出和损失，我们可以保存下来用于绘图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5000 5% (0m 4s) 2.7199 Markytan / Irish ✗ (Czech)\n",
      "10000 10% (0m 8s) 1.9880 Pereira / Italian ✗ (Portuguese)\n",
      "15000 15% (0m 12s) 1.3121 Si / Korean ✓\n",
      "20000 20% (0m 17s) 1.2234 Maceachthighearna / Irish ✓\n",
      "25000 25% (0m 21s) 1.2451 Torres / Portuguese ✓\n",
      "30000 30% (0m 26s) 2.5215 Guillory / English ✗ (French)\n",
      "35000 35% (0m 30s) 1.9661 Ton / Korean ✗ (Vietnamese)\n",
      "40000 40% (0m 34s) 1.1618 Zhurkin / Russian ✓\n",
      "45000 45% (0m 39s) 1.6795 Mendelssohn / Dutch ✗ (German)\n",
      "50000 50% (0m 44s) 0.0529 Antimisiaris / Greek ✓\n",
      "55000 55% (0m 49s) 4.4218 Oorschot / Scottish ✗ (Dutch)\n",
      "60000 60% (0m 53s) 0.4951 Jue / Chinese ✓\n",
      "65000 65% (0m 58s) 1.2338 Atiyeh / Arabic ✓\n",
      "70000 70% (1m 2s) 3.2467 Petersen / Dutch ✗ (Czech)\n",
      "75000 75% (1m 7s) 2.1980 Roles / Portuguese ✗ (English)\n",
      "80000 80% (1m 11s) 0.2940 Parent / French ✓\n",
      "85000 85% (1m 15s) 0.4599 Gomatos / Greek ✓\n",
      "90000 90% (1m 19s) 0.5917 Tze / Chinese ✓\n",
      "95000 95% (1m 23s) 1.5835 Matos / Greek ✗ (Portuguese)\n",
      "100000 100% (1m 28s) 0.7840 Oorschot / Dutch ✓\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import math\n",
    "\n",
    "n_epochs = 100000\n",
    "print_every = 5000\n",
    "plot_every = 1000\n",
    "\n",
    "\n",
    "current_loss = 0\n",
    "all_losses = []\n",
    "\n",
    "def time_since(since):\n",
    "    now = time.time()\n",
    "    s = now - since\n",
    "    m = math.floor(s / 60)\n",
    "    s -= m * 60\n",
    "    return '%dm %ds' % (m, s)\n",
    "\n",
    "start = time.time()\n",
    "\n",
    "for epoch in range(1, n_epochs + 1):\n",
    "    # 随机选择一个样本\n",
    "    category, line, category_tensor, line_tensor = random_training_pair()\n",
    "    output, loss = train(category_tensor, line_tensor)\n",
    "    current_loss += loss\n",
    "    \n",
    "    if epoch % print_every == 0:\n",
    "        guess, guess_i = category_from_output(output)\n",
    "        correct = '✓' if guess == category else '✗ (%s)' % category\n",
    "        print('%d %d%% (%s) %.4f %s / %s %s' % (epoch, epoch / n_epochs * 100, time_since(start), loss, line, guess, correct))\n",
    "\n",
    "    if epoch % plot_every == 0:\n",
    "        all_losses.append(current_loss / plot_every)\n",
    "        current_loss = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 结果绘图\n",
    "\n",
    "把所有的损失都绘制出来可以显示学习的效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fe2ac41a5c0>]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VOW9x/HPbzLZ95CFrIQ1AQIxEBAQ2VxAVGytWpeqbVWK0lZ7bau17W2tvbWttvW6112va1txQRDcQPYlhDWEJUAIIQlJIBskIdtz/5gBCWQZkkkmmfzer1deTeY8c87vOPSbk+c853nEGINSSin3YnF1AUoppZxPw10ppdyQhrtSSrkhDXellHJDGu5KKeWGNNyVUsoNabgrpZQb0nBXSik3pOGulFJuyOqqA4eHh5vExERXHV4ppXqlTZs2lRpjItpr57JwT0xMJCMjw1WHV0qpXklEDjrSTrtllFLKDWm4K6WUG9JwV0opN6ThrpRSbkjDXSml3JCGu1JKuSENd6WUckO9LtwPHavm4YVZ1Dc2uboUpZTqsXpduO8uquLV1bm8vT7P1aUopVSP1evC/ZLhkUwc1I8nvthDRU29q8tRSqkeqdeFu4jw6yuHU15Tz7PLclxdjlJK9Ui9LtwBUmKDuTYtjldX53LoWLWry1FKqR6nV4Y7wC9mJmGxwJ+X7HJ1KUop1eO0G+4iEi8iy0Rkp4hkici9LbQJFpGFIrLV3uYHXVPuN/oH+zB3ymAWbSskM6+sqw+nlFK9iiNX7g3A/caYEcAEYL6IjDirzXxgpzEmFZgG/E1EvJxaaQt+NGUQ/fy9eHzp7q4+lFJK9SrthrsxptAYk2n/vgrIBmLPbgYEiogAAcAxbL8UupS/t5X504ewZt9RVueUdvXhlFKq1zivPncRSQTSgPVnbXoaGA4UANuBe40x3fKU0c0XJhAT7MNfl+7GGNMdh1RKqR7P4XAXkQDgfeA+Y0zlWZtnAluAGOAC4GkRCWphH3NFJENEMkpKSjpR9jd8PD2479JhbD1Uzmc7jzhln0op1ds5FO4i4okt2N8yxixoockPgAXGJgc4ACSf3cgY84IxJt0Ykx4R0e4SgA67dkwsgyL8eXzpbhqb9OpdKaUcGS0jwMtAtjHm7600ywMusbePApKA/c4qsj1WDwv3X5bE3uLjfK5X70op5dCV+0XArcAMEdli/5otIvNEZJ69zSPAJBHZDnwJPGCM6dY7nDNHRhHq58mnOwq787BKKdUjWdtrYIxZBUg7bQqAy51VVEdYPSxcNiKKT7cXcbKhEW+rhyvLUUopl+q1T6i25IqUaKpONrAm56irS1FKKZdyq3CfNKQfgd5WluwocnUpSinlUm4V7t5WD2YMj+SznUU06GIeSqk+zK3CHWDWyP6UVdezIfeYq0tRSimXcbtwn5oUgY+nhaXaNaOU6sPcLtz9vKxMHRbB0qwjNOkDTUqpPsrtwh1gVkp/iipr2ZJf7upSlFLKJdwy3KcnRQKwdp8OiVRK9U1uGe4hfl4M6OfHjsMVri5FKaVcwi3DHSAlJpgdBRruSqm+yW3DfWRsEIeO1VBRXe/qUpRSqtu5bbiPig0G0Kt3pVSf5LbhPjLGHu7a766U6oPcNtzD/L2IDfFlu4a7UqoPcttwB0iJDSKr4OwVAZVSyv05shJTvIgsE5GdIpIlIve20m6afSGPLBH52vmlnr+UmGAOlJ6gqlZvqiql+hZHrtwbgPuNMSOACcB8ERlxZgMRCQGeBeYYY0YC1zu90g5IibP1u+vVu1Kqr2k33I0xhcaYTPv3VUA2EHtWs5uxLZCdZ29X7OxCOyJFb6oqpfqo8+pzF5FEIA1Yf9amYUCoiCwXkU0icptzyuuciEBvooK8NdyVUn1Ou2uoniIiAcD7wH3GmLP7OazAWOASwBdYKyLrjDF7ztrHXGAuQEJCQmfqdtio2GB2aLeMUqqPcejKXUQ8sQX7W8aYBS00yQeWGmNOGGNKgRVA6tmNjDEvGGPSjTHpERERnanbYSNjgtlXcpzquoZuOZ5SSvUEjoyWEeBlINsY8/dWmn0ETBYRq4j4ARdi65t3uZTYYIyBnXr1rpTqQxzplrkIuBXYLiJb7K89BCQAGGOeN8Zki8gSYBvQBLxkjNnRFQWfr1PTEGzLryA9MczF1SilVPdoN9yNMasAcaDdY8BjzijKmaKCvIkO9iEzr4wfMtDV5SilVLdw6ydUAUSEsQNCycgtwxhddk8p1Te4fbgDjEsMo6iylsPlNa4uRSmlukWfCPexA0IB2HSwzMWVKKVU9+gT4Z7cP5AAbysbc4+5uhSllOoWfSLcrR4W0hJCyMjVK3elVN/QJ8IdIH1AGLuPVFFRozNEKqXcX58J93GJoRgDmXl69a6Ucn99JtwvSAjBwyJs0q4ZpVQf0GfC3c/LysiYIL2pqpTqE/pMuIOt331rfjl1DU2uLkUppbpU3wr3xFBq65vIKtD53ZVS7q1vhbv9YSYdEqmUcnd9Ktwjg3wYFO7Pmn2lri5FKaW6VJ8Kd4DJQ8NZt/8YJxsaXV2KUkp1mT4X7hcPjaCmvpHMg+WuLkUppbpMnwv3CYPC8LAIq3JKXF2KUkp1GUeW2YsXkWUislNEskTk3jbajhORBhG5zrllOk+gjydjEkJYuVf73ZVS7suRK/cG4H5jzAhgAjBfREac3UhEPIC/AJ85t0Tnmzwkgu2HKyg7UefqUpRSqku0G+7GmEJjTKb9+ypsC1/HttD0J8D7QLFTK+wCFw8LxxhYraNmlFJu6rz63EUkEUgD1p/1eizwbeC5dt4/V0QyRCSjpMR1fd6jY4MJ9LGySrtmlFJuyuFwF5EAbFfm9xljKs/a/ATwgDGmzef6jTEvGGPSjTHpERER51+tk1g9LEwa3I+Ve0t1XVWllFtyKNxFxBNbsL9ljFnQQpN04F0RyQWuA54VkW85rcoucPHQCA6X13Cg9ISrS1FKKaezttdARAR4Gcg2xvy9pTbGmIFntH8N+MQY86GziuwKFw8NB2Dl3lIGRQS4uBqllHIuR67cLwJuBWaIyBb712wRmSci87q4vi4zoJ8/if38+CL7iKtLUUopp2v3yt0YswoQR3dojPl+ZwrqTleOjub5r/dTevwk4QHeri5HKaWcps89oXqmq1NjaGwyfLq90NWlKKWUU/XpcE/uH8SwqAAWbtVwV0q5lz4d7gBXj45hQ+4xCsprXF2KUko5jYZ7agwAi7bp1btSyn30+XBPDPdndFwwH28tcHUpSinlNH0+3MHWNbP9cIU+0KSUchsa7sBVqdEALNSrd6WUm9BwB6KDfRmfGMbHWwt0rhmllFvQcLe7+oIYcoqPk11Y5epSlFKq0zTc7Wan9MfDIizcpl0zSqneT8Pdrl+AN5OHhLNQu2aUUm5Aw/0Mc1JjyC+rITOv3NWlKKVUp2i4n+HykVF4Wy06akYp1etpuJ8h0MeTGcmRfLKtkIbGNheVUkqpHk3D/SxzUmMoPX6SdfuPuboUpZTqsHbDXUTiRWSZiOwUkSwRubeFNreIyDYR2S4ia0QktWvK7XrTkyMJ8Lby5Jd7WbW3lHq9gldK9UKOXLk3APcbY0YAE4D5IjLirDYHgKnGmFHAI8ALzi2z+/h4enDfpUPZfriC7728njGPfM7/frHX1WUppdR5cWQlpkKg0P59lYhkA7HAzjParDnjLeuAOCfX2a3uvHgQt1w4gFU5pby9/iD/+GIP05IiSI0PcXVpSinlkPPqcxeRRCANWN9GszuATzteUs/g6+XBZSOieOrmMYQHePGnxdk6/l0p1Ws4HO4iEgC8D9xnjKlspc10bOH+QCvb54pIhohklJSUdKTebhfgbeWnlwxl/YFjLNtd7OpylFLKIQ6Fu4h4Ygv2t4wxC1ppMxp4CbjGGHO0pTbGmBeMMenGmPSIiIiO1tztbhqfQGI/P/7y6W4am/TqXSnV8zkyWkaAl4FsY8zfW2mTACwAbjXG7HFuia7n6WHhFzOT2X2kivcz811djlJKtcuRK/eLgFuBGSKyxf41W0Tmicg8e5v/BvoBz9q3Z3RVwa4ye1R/UuNDeOLzPXr1rpTq8RwZLbMKkHba3Anc6ayieiIR4e6pg5j3ZiYr9pQwPTnS1SUppVSr9AnV8zAjOYrwAC/e3Zjn6lKUUqpNGu7nwctq4Ttj4vgyu5jiqlpXl6OUUq3ScD9PN4yLp6HJsCDzsKtLUUqpVmm4n6fBEQGMTwzjvY2H9KEmpVSPpeHeAd8dF8+B0hNsOKAzRyqleiYN9w6YPSqaQG8r72UccnUpSinVIg33DvD18uCatBgWbSvUG6tKqR5Jw72D7pw8iIYmw3PL97m6FKWUOoeGewclhvvznTGxvLU+j8KKGleXo5RSzWi4d8JPZgylqcnwzLIcV5eilFLNaLh3QnyYHzeMi+e9jYfIL6t2dTlKKXVau3PLqLb9ePoQ/pORz58WZzNlaAS7j1RRU9fI7+eMxMfTw9XlKaX6KA33TooJ8eXmCxN4bU0ui7cX4WW1UNfQxKQh4cxJjXF1eUqpPkrD3QkemJXMjORIBob7Ex3sw5S/LmNBZr6Gu1LKZbTP3Ql8vTyYMiyC+DA/rB4WvpUWy8q9pToGXinlMo6sxBQvIstEZKeIZInIvS20ERF5UkRyRGSbiIzpmnJ7h2vHxNLYZPh4S4GrS1FK9VGOXLk3APcbY0YAE4D5IjLirDZXAEPtX3OB55xaZS8zJDKQ0XHBfLBZZ45USrlGu+FujCk0xmTav68CsoHYs5pdA7xhbNYBISIS7fRqe5Fr02LJKqhkd1GVq0tRSvVB59XnLiKJQBqw/qxNscCZs2jlc+4vgD7l6tQYrBZhwWZdUFsp1f0cDncRCQDeB+4zxlR25GAiMldEMkQko6SkpCO76DX6BXgzLSmCjzYX6ILaSqlu51C4i4gntmB/yxizoIUmh4H4M36Os7/WjDHmBWNMujEmPSIioiP19irfToujqLKW1Tml52yra2hyQUVKqb7CkdEyArwMZBtj/t5Ks4+B2+yjZiYAFcaYQifW2StdOiKSED/Pc+Z9zyk+zthHPuellftdVJlSyt058hDTRcCtwHYR2WJ/7SEgAcAY8zywGJgN5ADVwA+cX2rv42314Ntpsby1Lo9jJ+oI8/cC4MUV+6k62cCfFmczIiaISYPDXVypUsrdODJaZpUxRowxo40xF9i/FhtjnrcHO/ZRMvONMYONMaOMMRldX3rv8N1x8dQ1Np0eFllcVcsHmw9zbVosA8P9+cnbm3XKYKWU0+kTql0suX8QqfEh/Mu+oPbra3Kpb2riJ5cM5Z+3plNb38jdb2ZysqHR1aUqpdyIhns3+G56PLuPVLF231HeXJfHzBH9GRjuz5DIAB6/PpUth8p5a12eq8tUSrkRDfducHVqNL6eHvz03c1U1NQzd+qg09uuGBXNkMgAlu0udmGFSil3o+HeDQJ9PLlydDSlx+tIHxDKmITQZtunDYtg/f5jVNc1uKhCpZS70XDvJrdcmICHRZg/fcg526YmRVDX2MS6/UddUJlSyh1puHeTtIRQMn97GdOTI8/ZNi4xDB9PC1/vdu+ndpVS3UfDvRsF+3q2+LqPpwcTB/Xj6z0a7kop59Bw7yGmJUWSe7Sa3NITri5FKeUGNNx7iKnDbHPtrNirV+9Kqc7TcO8hEsP9GdDPj+Xa766UcgIN9x5k6rAI1u47Sm29Pq2qlOocRyYOU91kWlIEb6w9yMMLd3L0+EmyCir53oQB3D1tsKtLU0r1Mnrl3oNMGNSPAG8r72zIY2/xcfy8PPjHF3vIL6t2dWlKqV5Gr9x7ED8vK1/ePxVvq4UQPy8KymuY/vhy/rpkN0/elObq8pRSvYheufcwUUE+hPjZ5n2PCfFl7pRBfLy1gMy8MhdXppTqTTTce7h5UwcTEejNHz/ZiTG6FqtSyjGOLLP3iogUi8iOVrYHi8hCEdkqIlkioqswOZG/t5VfXJ5EZl45L686oGuvKqUc4siV+2vArDa2zwd2GmNSgWnA30TEq/OlqVO+MzaO9AGh/HFRNhMf/ZL/WbSTgnJdvUkp1TpHltlbARxrqwkQaF9IO8DeVueudSIPi/Dejyby6vfHkZ4Yyqurc7ntlQ00Nmk3jVKqZc7oc38aGA4UANuBe40xLfYdiMhcEckQkYySEn0S83x4WITpyZH889Z0nropjZzi46fXZW3JsRN1bD1U3o0VKqV6EmeE+0xgCxADXAA8LSJBLTU0xrxgjEk3xqRHREQ44dB906yU/oyKDeaJL/a02AdvjGH+W5lc9/wa7b5Rqo9yRrj/AFhgbHKAA0CyE/arWiEi3H/5MPLLanhv47lrry7aXsja/UepbzS8tPJAq/tZvL2QyX/5iozctnrdlFK9kTPCPQ+4BEBEooAkYL8T9qvaMHVYBOMSQ3nqqxxq6r6Zi+bEyQb+Z1E2I2OCuOaCGN7ZkEfZibpz3r/lUDk/e28L+WU13PlGBvtKjndn+UqpLubIUMh3gLVAkojki8gdIjJPRObZmzwCTBKR7cCXwAPGmNKuK1mB7er9FzOTKa46yXPLc07fXH1mWQ6FFbX84ZqRzJ8+hJr6Rl5bk9vsvQXlNdz5egaRQd58cM8krBbh9lc2UFxV64IzUUp1BXHVgzHp6ekmIyPDJcd2J3e+nsEX2UeIDfHlqtRoXll1gDmpsfzthtTT2zfmHmPNgzPw97ZSXl3HTS+uJ/9YNQvumcTQqEC2HirnxhfWMTjSn3/9aCJ+XjorhVI9lYhsMsakt9dOn1Dt5Z69ZQzP3DyGQRH+vLBiPz5WDx64Iun09numD6aipp7nv97HY0t3cfFflrHnSBVP3ZzG0KhAAFLjQ3jmljR2HK7kmWU5rjoVpZQT6SVaL+dltXDl6GiuHB3N4fIa6huaiAz0Ob19TEIoEwaF8dRXOYjA7JRo5k8fwoiY5gOaZiRHcc0FMby48gA3jksgPsyvu09FKeVEGu5uJDbEt8XXf3f1SP6zKZ+bxicwJDKg1fc/MCuZpVlFPPppNs/eMrarylRKdQPtlukDhkcH8durRrQZ7GCbhfLuqUNYvL2IdfuPdlN1SqmuoOGumpk7ZRAxwT78YeFOnd5AqV5Mw1014+vlwYOzh7OzsJJF2wtdXY5SqoM03NU5rhoVTXyYL++sP/fpV6VU76Dhrs5hsQjfTY9n7f6j5JaeaLYtu7CS2vrGVt6plOopNNxVi65Pj8fDIry78dDp1zYdPMYV/7uShz7Y7sLKlFKO0HBXLYoK8mF6UiT/2ZRPfWMT9Y1N/PoD22JcH2w+zM6CShdXqJRqi4a7atVN4+MpPX6SL7OLeXX1AXYVVfHYdaMJ8vHkz0t2ubo8pVQb9CEm1aqpwyLoH+TDc8tz2HPkOJcOj+L69Hgqaur546JsVu0tZfLQcFeXqZRqgV65q1ZZPSzckB7H1vwKAH4/ZwQAt04cQGyIL49+mk2TjoVXqkfScFdtumFcPN5WC/dfPoy4UNt8M95WD34xM4msgkre2tD2cMmnv9rLbz/ccc4vgSe/3MvMf6zgz5/uYsuhclw1O6lS7kq7ZVSb4kL9yPjNpQT6eDZ7fU5qDP/edIjffriDmroG7rp4ELY10r9RW9/Ic8v3caKukWBfT34+0zZb5UdbDvP3z/cwKNyfl1bu5/mv9zE6LpgFd0/C6qHXG0o5gyOLdbwiIsUisqONNtNEZIuIZInI184tUbna2cEOtrHwL98+jitHRfOnxbt4uIXpClbsKeFEXSOj44J5elkOC7cWsD2/gl/+ZxvjE8NYct8UNv3mMu67dCjb8itYf0CX+1PKWRy5THoNmNXaRhEJAZ4F5hhjRgLXO6c01dP5eHrw1E1p3Dl5IK+tyeXPn2Y32/7pjiJC/Dx5d+4ExiWG8vN/b+WO1zcSHuDNs98bg5fVQrCfJz+aMhhfTw8W63QHSjlNu+FujFkBtHVJdTO2BbLz7O2LnVSb6gUsFuE3V43gurFxvL724Oml+k42NPLFziNcPiIKPy8rz31vLOEB3lTVNvDCbbbvT/H18mDG8EiWZhXpZGVKOYkzOjiHAaEislxENonIba01FJG5IpIhIhklJSVOOLTqKeZPH0JDYxOvrMoFYHVOKVUnG7hiVDQA4QG29VoX/mQyI2OCz3n/7JRoSo/Xsf6ATjWslDM4I9ytwFjgSmAm8FsRGdZSQ2PMC8aYdGNMekREhBMOrXqKgeH+XDk6hjfXHaSiup7F24sI9LFy0eBvxsFHBvm0Oqf89OQIfDwtfLq9qLtKVsqtOSPc84GlxpgTxphSYAWQ6oT9ql7m7qmDOX6ygZdX7eezrCIuGxGFl9Wxf2J+XlZmJEfy6Q7tmlHKGZwR7h8Bk0XEKiJ+wIVAdjvvUW5oREwQM5IjeWb5PiprG5idEn1e778iJZrS4yfZmOucUTNVtfX6i0L1WY4MhXwHWAskiUi+iNwhIvNEZB6AMSYbWAJsAzYALxljWh02qdzb/OmDaWwyBHhbuXjY+U1NMCM5Em+rxSmjZqrrGpj62HJeXLm/0/tSqjdq9yEmY8xNDrR5DHjMKRWpXm3sgDCuGh1NbIgv3laP83qvv7eV6Um2rplfXzn8vN9/ps93HuHYiTpW7S1l3tTBHd6PUr2VPg6onO7pm8fwq9nDO/Temy5MoKTqJA8v3NmpGj7eUgDAlkPl53TNPL50N/+3NrdT+1eqp9NwVz3K1GER3D1tMG+vz+OdduataU3ZiTq+3lNCXKgvx082sLe46vS26roG/rliH498ks2Bs1aZUsqdaLirHufnlycxdVgE//3RDjYdLGu2zRjDwq0FfOe5NWQXtrxgyOIdhTQ0GR6y//WQebD89LYNB45R32iob2ri9x9nnfeEZXuPVPH6mlyd6Ez1eBruqsfxsAhP3phGTIgvc9/I4O+f7WbroXIKymu4640MfvLOZjYdLOM3H+5oMWQ/2lLAkMgArkjpT5i/F5l53/yCWJ1TipeHhZ9fnsTXe0r4fOcRh+sqqqjley+v53cfZ5FdWNX+G5RyIQ131SMF+3ny8u3pDI4M4OllOVzzzGom/fkrVuWU8uvZw3n02lFsOljGh1sON3tfQXkNGw4c45rUGESEMQkhbD4j3FfuLSU9MZS5UwYxLCqAP3yy06EFv6vrGrjzjY0cr23AIrA0Sx+2Uj2bTvmreqwhkYH860cTKTtRx/I9xewqquLm8QkM6OdPU5Ph3Q15PLp4F5eN6E+At+2f8sKtthupcy6IASAtIZQvsospr66jvtGwq6iKX8xMwtPDwsNzUrjpxXXc9UYGo2KD6RfgzYRBYedMj9DUZPiv97aSVVDJS7el88+v97M0q4ifXdbig9idUlFTz9IdRVyQEMKwqECn71/1HRruqscL9ffi22lxzV6zWITfzxnJt59dw1Nf7eVXVwzn0LFqFmQeJjU+hAH9/AFISwgBYPOhcipr6gGYPMQ2/n7i4H7cPW0w72/KZ+2+ozQ0GeJCfVn5y+nN5qZ/Y20uS7KK+M2Vw7lkeBQHSk/wx0XZHDx64vRxOiu7sJKXVx3gk20F1NY3MXVYBK//cLxT9q36Jg131WulJYRy3dg4Xll1gE+2FnK4vAaAx64bfbpNalwIFoHNB8sorKgl2NeTlNhvrswfmJXMA7OSMcbwyupcHvlkJwePVpMY/k1of7KtkJExQdwxeSAAM0f254+LslmaVcTcKZ0fQ19b38gN/1xLU5Ph22lxFFfWsv7AMRqbDB4WaX8HSrVA+9xVr/bLWUmkxAaTGh/MH64ZyWc/m8L16fGnt/t7W0nuH0RmXjmrc0qZNLhfi4EpIkxPsk1mtzKn9PTrlbX1bD5UzrSkiNNX8/FhfoyIDmJpluM3Y9uyMfcYVbUNPHVzGo9eO4o5F8Rw/GRDq6OBlHKEhrvq1SIDffjgnot49pax3DYxscV+6jEDQli3/ygFFbVcNKT1KREGhvsTG+LL6r3fhPuanFIamwxThjafxXTmyP5k5pWdnr++M5bvLsHLamHiIFtt4weGAejKVKpTNNyV20uLD6XB/pTqxUNbD3cR4aIh/Vizr/T0U61f7yklwNvKmAGhzdrOTInCGM5rKGVrlu8u5sKBYfh62aZbiA72JSHMjw06t73qBA135fZOBXNcqC002zJ5aASVtQ1sP1yBMYYVe0qYNLgfnmct3J0UFciAfn6d7po5dKyafSUnmJYU2ez18QPD2HDgmD4spTpMw125vcR+fsSG+HLp8Khmo2BaMmlwP8D2sNP+0hMcLq9hyrBzF5YREWaO7M/afaUcPX6yw7Ut32NbkWxaUvNjjB8YRll1PTnFxzu8b9W3abgrtyciLPzJZB68IrndtuEB3gyPDmLV3lJW2IN3agvhDnBDehyNTYZnl+87Z5sjD0YBfL27mPgwXwaFNx9SeaH2u6tO0nBXfUKYvxc+no5NIXzx0HA2HSxjaVYRA8P9iW+lK2dIZCDfGRPH/609SH5Z9enXP9lWQMrvlrLlUHmL7zvlZEMja/YdZdqwyHP+okgI8yMqyJsNGu6qgxxZrOMVESkWkTYX4BCRcSLSICLXOa88pbrfRUPCqWtsYt3+Y0xp4wYsYHtKVeAfn+8FbBOL/fI/22hoMizZ0fYUBRm5ZVTXNZ7TJQO2vzbGD+zXar97Q2MTG3OP8dLK/fz47Uwe+mC79s+rZhy5cn8NmNVWAxHxAP4CfOaEmpRyqfGJYXjZb6C21N9+ppgQX74/KZEFm/PJzCtj3pub8PPyYHh0EF/bu3VOOXr8JGMf+ZzbX9nAlkPlLN9djJeHhYn2fv5z6wilqLKW/LKac7b97F9buf75tfxxUTZr9h3l7fV5fOaEkTvKfbQb7saYFUB7fxv+BHgfKHZGUUq5kq+XB2MHhOLpIUwY1HLwnumeaYMJ8LZy4wvrOFB6gqduGsOc1BiyCys5UvnNOPhF2ws5eqKOzXllfOuZ1byx9iDjB4bh59Xyg+LjB9qOfXa/+9ZD5SzcWsAPLkpk468vZcNDlzAowp/Hlu6mobHJoXM8cbLBoXaq9+p0n7uIxALfBp7rfDmLQLjlAAAQgElEQVRK9Qz/dfkw/vitFPy925+hI8TPi3lTB1PX0MQvZyUzcXC/010tZ169f7ylgKSoQNb86hJ+fvkwgnw9uXZMbKv7HRoZQIifJ1/tan5F/vhnuwnz9+L+y5OICPTG6mHhF5cnkVN8nAWZh1vZ2zdeX5NL6sOftdltZIxtYrYzfzmp3sUZN1SfAB4wxrR7ySAic0UkQ0QySkpK2muulMuMSwzju+MSHG4/b+pgPrhnEj+aMgiA5P6BRAZ6nw73/LJqMg6WMeeCGAK8rfx4xlA2/vpSrh0T1+o+LRbhpvEJLN5exOtrcgFYs6+UlXtLT/+1cMqslP6kxofwjy/2tDlS56tdR3h4YRYi8OCCbRRVtBzeS3YU8eCC7byxNtfh/waqZ3FGuKcD74pILnAd8KyIfKulhsaYF4wx6caY9IiItvsylepNPCxCWkLo6VEvIsLUYRGs3FNCQ2MTC7cWAnD16Jjz2u/PL0/ishFRPLwwiy92HuHxpbuJDvbhexMGNGsnIjwwK4nCilr+b+3BFve1s6CSn7y9mRExQXxwz0WcrG/i/n9voemsNWZr6xv506fZAGzLrzivejvj7DpU53Q63I0xA40xicaYROA/wD3GmA87XZlSvdy0pEgqaxvYml/Ox1sLSEsIIaFf20/Ins3DIvzvjRcwKjaYeW9uIjOvnJ9eMrTFYZ2TBoczZVgEzyzPocI+vfEpxVW13PH6RoJ8PXn59nGkxAbzu6tHsDrnKC+t2t+s7Wtrcjl0rIakqEC25Ve0OgqnvrGJ+W9n8vjS3Q6fz5IdRTy2dBf1Z90beGnlfi589MtmQ0rPduJkA/e+u5mDR3XtW0c4MhTyHWAtkCQi+SJyh4jME5F5XV+eUr3X5CHhWAReWnmA7MJK5qSe31X7KX5eVl66fRz9g30YFO7PdWNb78r55cwkyqvreXFF88D+06Jsjp6o4+XbxxEV5APAd8fFM3NkFI8t3c2b6w7S0NhESdVJnv4qh0uHR3L7pEQqauo5dOzc0ToAjy7exaJthTy9LIflu5uPpdhyqJwFmflUVNt+yVTXNfDg+9uY9+Ymnlm2j5++s/l0wH+8tYA/LsqmpOokr67ObfXcPttZxEdbCnh7fccWTu9r2r1bZIy5ydGdGWO+36lqlHIjwX6epCWE8umOIiwCV46O7vC+IgK9WXLfFBoam86Z5+ZMKbHBXJ0aw8urDnD7pEQiAr3ZdPAYH24p4MfThzAiJuh0WxHhz9eO5kdvbuI3H+7g1dUHiAnxpba+kYdmD6e6ztZ3vzW//Jy/OD7acphXVh/glgsT2Jh7jAfe38Zn900l2M+TNTml/PD1jdTWN+HpIUweEk7esWr2l57g7mmDCfXz5E+Ld/Gz97Zw8/gEfv6vrYwfGEZ4gBfvbTzEvZcOJcjH85xzO3UDeElWEQ9ekdzuVBJ9nT6hqlQXmmYfJz9xcD8iA306ta8Abyshfl7ttrv/smHUNzbx9Fd7aWoyPLxwJ/2DfLhn+rkLi4T6e/He3Am8cOtYjLGtMXvbxEQGRQQwLCoQL6uF7Yeb97tnF1bywPvbGJ8Yxu/njORv11/A0eN1/O7jHaeDfUCYP2/deSE/uGgge44cp7a+ibfuuJAHZiUzd8pgHpqdzCfbCrnl5fXEh/nywq1juWfaEI6fbODdDedemVfXNfD1nhLCA7w5eLSa3Ud0gfL26EpMSnWhGcMj+dvne85ZJrArJYb7c8O4eN7ekEewryfb8it44rsXtDqeXkS4fGR/pidHsjqn9PTYfi+rheHRQWzL/2YaBWMMP31nM0E+njx9SxqeHhZGxQXz4xlDeOKLvSzebpuy4a27LiQ8wJuLhoTzqxausudOGYwgvJ+Zz4u3pRPi50WInxcTBoXx6upcfnDRwGZ/oazYU0JtfRN/vW4E9767mSU7ikjuH0RH/XXJLg4erea/Lh/G4IiADu+nJ9Mrd6W60MiYYD7/2RS+08Z49q5w7yVDsYjw5Fc5jEkI4ZoL2u/v9/SwMC0pstnN2tS4YHYcrjw9kiUzr4y9xcf5xcykZn+JzJ8+hDEJIQyK+CbYT2mt++SuKYNYct+UZnP33HXxIAoralm8vbBZ26VZRwj182R2Sn/SB4R2aqrldzbk8ezyfSzJKmLmP1bw+4+zKDtR1+H99VQa7kp1saFRgd3ePxwV5MMdkwdiEfjd1SM7fPxRscEcP9nA/lLbCJUFmYfx8bRwxajm9w88PSz860cTWfzTi5sF+/manhTJoAh/Xlp54PQonbqGJr7IPsKlw6OweliYObI/2YWV5B1tfWRNazbnlfG7j7K4eGg4ax6cwfXp8byxNpernlrVqambeyINd6Xc1M8vT2LVAzNIjQ/p8D5Gx9neuy2/nLqGJj7ZVsjlI/o3e4DqFKuHBUsnF/S2WIS7Lh7E9sMVPPe1bSrltfuPUlXbwKyU/oBtiUOApVltT8wGcLi8hnX7j5J3tJqC8hrufjOTyCBvnrwxjaggHx69dhT/njeR0uMnmf925jlDNHsz7XNXyk1ZLEJMiG+n9jEkMgBfTw+25Vfg722loqaeb3dxF9MN6fGs23+Uvy7ZjSDkHavG38vj9Pq38WF+jIwJYklWEXfZnwhuyYHSE1z15EpO1H3zxK631cKCeyYR6v/NjemxA8J49NpR/Ne/tvI/i7L5/ZyRXXdy3UjDXSnVKg+LkBIbxPbDFRRV1BIe4MXFbSwy7qxj/u36VAD+smQXXh4WLh8Z1exewMyR/fnHF3sorqwlMujcUUi19Y3MfysTT6uFF29Mo6y6jsLyWiYN6cfImOBz2l87Jo4dhyt5ZfUBBkcGcOO4+DaHnJ5ijKGk6mSLNZxSXdfA/Lcy+eHkgVw8tPuezNdwV0q1aXRcCG+uO4gxcMuEBKwOhF5nWT0spwP+oy0FXJHSvI9/Vkp//v75Hq59bg1XjY7hqtHRjIwJOn1v4U+Ls9lZWMlLt6Vz6Ygoh4750OxkdhVV8tsPd/CXT3cxYVAYM0f257qxcS3esyipOslDH2zn851HeP57Y5iV0vJzDC+s2M+y3SUcO1HH5CHh3Xb/RcNdKdWm0XHBnGyw9UVf241DOk8F/K0TBjDWvsj5KcOiAnn+e2N4Z8MhXlq5n+e/3kdsiC/TkyOICvThjbUHuXPyQIeD/dTxXvn+OL7aVczqnFJW5ZTyRfY2DpfXcN+lw5q1XbStkN98uJ0TdY1EB/vwh4U7mTIs4pzhpoUVNTz/9T7CA7zYml9BZl75OefSVTTclVJtOnVTdXCEPymxHR9b3hFWDwvpiWEtbpuVEs2slGjKTtSxNKuIL3cVsyDzMNV1jaTGBfPLWe2vmXs2H08PZo+KZvaoaJqaDL98fxtPfLGXQB9P7pg8kILyGh5emMXSrCOkxgXztxtSKauu5/rn1/LMshx+MbP5Mf+6ZDdNBt6+awLXPbeGV1Yd0HBXSvUMA8L8GB0XzI3jEnrkI/+h/l7cOD6BG8cncLKhkc155aefru0Mi0X487WjOHGygUc+2cmuwkoWbS+kyRgemJXMXRcPPN1FdW1aLC+uOMB1Y+MZaF/sfHNeGR9sPsz86YMZFhXITeMTeHHlfvLLqokLPb8J5DpCXLXuYnp6usnIyHDJsZVSylEnGxq5641NrNhTwvSkCP5wTco5i6YXV9VyyeNfkzYglAdnJZN3rJpnluVQVFnLsp9PI8DbyuHyGqb8dRl3Th7Ir2YP73A9IrLJGJPeXju9cldKqTZ4Wz148bax7Ck6TkpsUIt/vUQG+nDfZcN45JOdrLAv0OJhEf7x3QtOPxMQG+LLrJT+vLMhj59eMtShVb46Q8NdKaXa4W31YFTcuUMoz3T7xAEEelsJ8LGSEOZHfJgfwb7NZ7f84UUDWbStkPcz87ltYmIXVqzhrpRSTmH1sHDDuPg224xJCGFOaoxDs3t2liOLdbwiIsUisqOV7beIyDYR2S4ia0Qk1fllKqVU7yciPHlTWocXbjkfjtxOfg2Y1cb2A8BUY8wo4BHgBSfUpZRSqhMcWYlphYgktrF9zRk/rgO67ykHpZRSLXL2c8R3AJ86eZ9KKaXOk9NuqIrIdGzhPrmNNnOBuQAJCQnOOrRSSqmzOOXKXURGAy8B1xhjjrbWzhjzgjEm3RiTHhHRfbOjKaVUX9PpcBeRBGABcKsxZk/nS1JKKdVZ7XbLiMg7wDQgXETygd8BngDGmOeB/wb6Ac/an9xqcOTRWKWUUl3HkdEyN7Wz/U7gTqdVpJRSqtNcNnGYiJQABzv49nCg1Inl9BZ98bz74jlD3zzvvnjOcP7nPcAY0+5NS5eFe2eISEZf7Prpi+fdF88Z+uZ598Vzhq47765fL0sppVS303BXSik31FvDva/OX9MXz7svnjP0zfPui+cMXXTevbLPXSmlVNt665W7UkqpNvS6cBeRWSKyW0RyRORBV9fTFUQkXkSWichOEckSkXvtr4eJyOcistf+v92zjHo3ExEPEdksIp/Yfx4oIuvtn/l7ItL1Kx10IxEJEZH/iMguEckWkYl94bMWkZ/Z/33vEJF3RMTHHT/rltbEaO3zFZsn7ee/TUTGdPS4vSrcRcQDeAa4AhgB3CQiI1xbVZdoAO43xowAJgDz7ef5IPClMWYo8KX9Z3d0L5B9xs9/Af5hjBkClGGboM6d/C+wxBiTDKRiO3e3/qxFJBb4KZBujEkBPIAbcc/P+jXOXROjtc/3CmCo/Wsu8FxHD9qrwh0YD+QYY/YbY+qAd4FrXFyT0xljCo0xmfbvq7D9nz0W27m+bm/2OvAt11TYdUQkDrgS20R0iG1OixnAf+xN3Oq8RSQYmAK8DGCMqTPGlNMHPmtsT8j7iogV8AMKccPP2hizAjh21sutfb7XAG8Ym3VAiIhEd+S4vS3cY4FDZ/ycb3/NbdkXSkkD1gNRxphC+6YiIMpFZXWlJ4BfAk32n/sB5caYBvvP7vaZDwRKgFftXVEviYg/bv5ZG2MOA48DedhCvQLYhHt/1mdq7fN1Wsb1tnDvU0QkAHgfuM8YU3nmNmMb5uRWQ51E5Cqg2BizydW1dCMrMAZ4zhiTBpzgrC4YN/2sQ7FdpQ4EYgB/2l7O02111efb28L9MHDm8uJx9tfcjoh4Ygv2t4wxC+wvHzn1J5r9f4tdVV8XuQiYIyK52LrcZmDrjw6x/+kO7veZ5wP5xpj19p//gy3s3f2zvhQ4YIwpMcbUY5s2/CLc+7M+U2ufr9MyrreF+0ZgqP2Ouhe2GzAfu7gmp7P3M78MZBtj/n7Gpo+B2+3f3w581N21dSVjzK+MMXHGmERsn+1XxphbgGXAdfZmbnXexpgi4JCIJNlfugTYiZt/1ti6YyaIiJ/93/up83bbz/osrX2+HwO32UfNTAAqzui+OT/GmF71BcwG9gD7gF+7up4uOsfJ2P5M2wZssX/Nxtb//CWwF/gCCHN1rV3432Aa8In9+0HABiAH+Dfg7er6nHyuFwAZ9s/7QyC0L3zWwMPALmAH8H+Atzt+1sA72O4r1GP7S+2O1j5fQLCNCNwHbMc2mqhDx9UnVJVSyg31tm4ZpZRSDtBwV0opN6ThrpRSbkjDXSml3JCGu1JKuSENd6WUckMa7kop5YY03JVSyg39P/OB8uGKmGwMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.ticker as ticker\n",
    "%matplotlib inline\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(all_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评估效果\n",
    "\n",
    "为了查看模型的效果，我们需要创建一个混淆矩阵，每一行代表样本实际的类别，而每一列表示模型预测的类别。为了计算混淆矩阵，我们需要使用evaluate方法来预测，它和train()基本一样，只是少了反向计算梯度的过程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVwAAAEwCAYAAAD7IMkNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXecXFXZx7+/3SQEAgQwiEgLHakBAtKliahIUaQqBlTQV6ovCqgvIBYUC9KUJkWlo0JEpBh6T6UkVANIEwk9EFJ2f+8f50z27uy0O5ndmd2cbz73k7nnPvfcM3dnzpx7zvP8HtkmkUgkEr1PW7MbkEgkEgsLqcNNJBKJPiJ1uIlEItFHpA43kUgk+ojU4SYSiUQfkTrcRCKR6CNSh5tIJBJ9ROpwE4lEoo9IHW4ikUj0EanDTfQrJC0n6feS/hH315X01Wa3K5GohdThJvoblwA3Ax+N+08BRzetNYlEDlKHuwBIWqzZbVgIGWH7aqATwPY8oKO5TUokaiN1uHUgaStJ04An4v5Gkn7b5GYtLLwn6UOAASRtAbzd3CYlErUxqNkN6KecDnwKGAtg+2FJ2zW3SQsN3ybc99Ul3QssC+zd3CYlErWROtw6sf2CpGxReqztA2xPkvQJYG1AwJO25za5WYlETaQphfp4QdJWgCUNlnQs8HizG7UwIOmLwKK2pwJ7AldJ2qTJzUokaiJ1uPXxDeBbwArAS8CouJ/off7P9ruStgF2An4P/K7JbUokakIp40OiPyFpsu2NJZ0KPGr78kJZs9uWSFQjjXDrQNJpkpaM0wnjJL0m6UvNbtdCwkuSzgP2BW6UtAjpc5zoJ6QPan3sYvsdYDfgOWAN4DtNbdHCwz6EwIdP2X4LWIZ07xP9hNTh1kfBu+OzwDW2kx9oH2H7feC/wDaxaB7wdPNalEjUTupw6+MGSU8AmwLjJC0LfNDkNi0USDoJOA44IRYNBv7UvBYlErWTFs3qRNIywNu2OyQNA5aw/Z9mt2ugI2kKsDEwqbBQJukR2xs2t2W1EcPB/xdY2fbXJa0JrG37hiY3LdEHpBFuHcQvzf/Q5Y70UWB081q0UDHHYZRQCO0d1uT25OViYDawZdx/Cfhx85qT6EtSh1sfFwNzgK3ifvrS9B1XRy+FpSR9HfgncEGT25SH1W2fBsyF+XPSqnxKYqCQQnvrY3Xb+0raH8KXRkVxvonewfYvJX0SeIcQ3nui7Vub3Kw8zJG0KF0j9NUJI97EQkDqcOsjfWmaiO1bJT1I/PxKWsb2G01uVq2cBNwErCTpMmBrYExTW5ToM9KiWR3EEdYPgHWBW4hfGtt3NLNdCwOSDgN+SPAK6SQ8jtv2ak1tWA6ivOQWhLY/YHtGk5uU6CNSh1sn6UvTHCQ9DWyZ935H7YU1bV8c3fgWt/1sA9u1ArAKmadG23eVsNsamGL7vRiduAlwhu3nq9TfDixXVP+/G9T8RB+ROtw6KKd9W+oLlqhO7AC/Doyke4dySAnbm4DPx8WmWus/ieBFsrbttSR9lBCwsvWCtj3W/3NCqPE0umQ6bXv3EraPABsBGxIWX38P7GP7ExXqP4IwFfEqMdNFrL+hrnC1/mgk6ifN4dZHNpR0KLA5MBHYsTnN6fdcD9xN8Diopit8AnBfnMOdP29u+8gK5+xF9N2Nti9LWmKBWtydPQmdeS3z+PNsW9IewDm2f19DEsyjYv2vL3BLy1DuRwNIHW4DSR1uHdj+XHZf0krAb5rUnIHAYraPq9H2POA24FG6RnvVmBM7ud7y3Z1OiHirpcN9V9IJwJeBbSW1xXMr8QK9n0Yoz49Gok5Sh9sYXgQ+1qyLR8WsL9DzkfyUZrUpJzdI+oztG2uwHWz72znrL/bdPYTG+u6+D0yRNI7qo+59gQOAQ2z/R9LKwC9KVSqp8D6nA3dI+ntR/b9uUPsL16j1RyNRJ6nDrQNJZxFdwgjBI6OIj6sVztmKnh3iHxrUpOsJI6CJ9KMvjKR3CfdRwPckzSYEBBQ8D5Yscdo/JB0K/I3unU9Zt7A+8N0dG7eqxE72z8CasWgG8Ncy5oVpj3/HbUjceoM8PxqJOkmLZnUg6SuZ3XnAc7bvrWD/R2B1YArdF1Ua8mGW9Jjt9RtRV6sjqZRnQUW3sDiF8EHUvVib0On+oxm50OII+1BgGdurRy2Fc23vVOP5bQQPi3dqsK35R77oMz0f25fW0q5EbaQOtw+Q9DiwrnvpZks6HzjL9qO9UX9vU8ZV6jeNcnuSNBHYFlgauAeYQJjXPbBB9a8JnErwyx5aKC/1IxDFdzYHHsyI7zxqe4MK9V9OSOvUAYwHliS4kpWciojn9OqPfKI+0pRCHUh6lK4phQJvE77IPy6xmvwY8BHglV5qxyDgYEnTCY+DhUfyfqGgRRAB2kjSRgQlrQuBPwIlXaUkrU/Pzq3S9Ixi+PVXgd/ZPi12fKXqHgbMst0Z99uAoVXc0C4muG2dDuwAHEx5nZLZtucUIsElDaLnZ6mYdW2/I+lA4B/A8YTpo7IdLsENruYf+Tw/Gon6SR1uffyDMGq4PO7vBywG/Ae4BPhckf0IYJqkh+g+P9bDTzMnu9V7Yov5XGZdpc6u5CoVfWq3J3QMNwKfJoxaK3a4krYEDgQK9baXsR0H7AzMjPuLEaIJtypjDyGL8DhJigEMJ8dR9YklbO+U9D1g0Tiv/D+E+ehKDJY0mOBJcLbtuQWPiwrk/ZHP86ORqJPU4dbHzrazqbkflTTJ9iZlcpudXGvFkj5t+x9FZd+wfW6xbSE6KWo5vGh7tqTtCU71ZTugFvS5LLhKfQnYroqr1N6EwIHJtg+WtBzVBciPJvjv/tX2VEmrAbeXsR1qu9DZYntmlOOsxOzY5qclHU5Qj1u8jO3xhE7/UeAwwo/GhVXqP4+Qyulh4C5JqxAWAHsg6W+Ev+US5PuRz/OjkSLf6iR1uPXRLmlz2w8BSNqMrhHTvGJj23fmqPv/JM22fVus+7uEEUePDjfDn4HRktYAzid4LVwOfKaMfS6fS0mfB34OfJgwXVHJi6AeCq5SX63mKkV83Jc0T9KShHQ7K1WqPN7/Owsdp+3pQLm5zPckbWJ7EoCkTYFZVdp/FGEkfCTwI0IATLlFqE6CS1rNbmm2zwTOzBQ9L2mHMua/rLXeImr+0SgX+Ub4oS/JAHBdbAhp0awOYgd7EeEDKcJo42vAVOCztq8ust8COIvgqzuE0Dm/V6rDkjQCuIEQzbYrsA6wv+05FdpTGF1/l9AhnaUKqcMl/QP4YnYkV+X9PgN8zvbjtdj3JpJ+C3yPMI3zv4RH/ym2D65wzpaEENrFba8c54oPs/0/JWw3A64EXib8bT8C7Gt7YoPavzXhiacwnVNWfEfSl2z/KeOP241KfriSVgVesf1B3F8UWM72c2XsNwMeB5Yi/GgMB06z/UAJ22eAj+eJfFMIyS64Ls6PJrT9q1rrGAikEW4d2B4PbCBpeNzPRgFdXeKUswkdxDWExYyDgLXK1D1D0u6EMNeJwN41LHzMVdDmPYiu+eNK0Ut5fS5f7Y3OVtI9trfJ+OPOP0SZEXSmkzw3fomXtP1IlUv9BvgU0VfW9sMqr4cxXtI6BNcxgCeruY9JGg18n55z4qVGfL8HjqGo4ylDISKuVBhytc/ENXSfd+6IZZuVMo6faQg/YGV/vCL1RL6taHvXnOcMOFKHWweSTizaByo/Htl+RlK77Q7gYkmT6UqESIlOZwiwGrB3mFar+Ph+MMFt6Ce2n42jmz9WsK/JUT9OJQBMkHQVcB3dO+i/VKujEra3if/n0jUoXvCTtF21BT/bL6i7Rny3zk7SjrZvy7znAmtJqvZeLyM8kdQSbvx28Rx9hTafF1/+00V+3nGkXIlB2aei6BnRI2hC0m9sH52Z+y1uw+4Z2wWJfLtP0gb91XWxUaQOtz7ey7weSvAWqDQCfD9+2KdIOo2wctxtBThvp1N07jQyc5IOsoM/r2BfqzN71tvifWCXbDXAAnW4WWpdhKlzwe8FhSAAx9X+o+j59/oEQaOh2MOkUH+l9/qa7ZoizYDbJf0i1pftrCpFKp5F8E2uVtatTZJ2L7QreoCUkrQs/DDXMvebO/JNA8d1sSGkOdwGEBcEbra9fZnjqxAWGIYQHieHA7+1/UwJWxHcl1a1/SMFYZzlCwt0RbZX295Hpf2Cyz3StpzPZblFmFLtl/QksGGtC37xnBHAGQR3LxHcvI7KMwdZpf6dgP0JLmUVnwAklfKOsO0eSnNx7nkrgpfF6ZlDSwJ72d6oQptWJ4y8P0p4zy8AB5X6zJU4d2lgpRqmaqoSP/tlcRUd4IFGGuE2hsWAFcsdtP18XLRY3vYPq9T1W0KnsyNh8WImcA6l596Oiv/n9cfN5XMp6VJCB/VW3F8a+JVL6NXWSR75wdwiKw5i5TVFlSkIy58EbEP4EbsHOKVK2w4mLG4OpvuqfY8O13Y574JSDCEszA6i+zzuOwT3uLLY/hewhaTF437FBVJJdwC7x2tNBP4r6V6XEAqSdCth0TX7ebjS9qdKtKMu18WBSupw66BoRNkOLEvoHMvZf47wyDYEWFXSKMKXuJRP5Mejx8FkANtvlpp7i8deif/nHSXk8rkkjCjfylz3TUklPSDqJM8iTG6RFeUQOCd4KNxFcGGC0FFfRRgdl2Mz22tXOF7cns8C69H96aLH/H/GnW2WQ6bfbB1fBJ6ucI1ublg1rDMMd4hm+xrwB9snKYill2LZEp+HD5drSySv6+KAJHW49ZEdUc4jrOL38L/NcDIhfv4OANtT4sJWKebG+cyCduuylFmIKbHQNv8Qlf1k8zjqA7RJWtr2m/G6y1Dhs5Ozg4N8izClFvyqzYvlEThf3nb2x/PHkvatcs59ktaNc+kVkXQu4YloB0LAw95Aj+miIvYDTisqO4HgdVCOvApygyQtD+xD8LioRIeklQtz7HHaoNrfoNP2vLgoeZaj62IN7RpQpA63Pn5s+8vZAkl/LC7LMNf220Wr5OU+oGcS5PqWk/QTwhfyB6UMF2ChrWZH/civgPslFb7gXwR+WsE+TwcH+eQHl7J9RrZA0lHljCN5BM5vkbQfXe59ewM3VzlnC8Ko+1mqLwhtZXtDSY/Y/qGkXxFCxXsg6dOEEeAKkrKBD0tSIsCmiLxuWKcQ3uc90TVuNcqPoL8P3CPpTsJ73ZaggFaJvK6LA5K0aFYHioEGmf1BwCO21y1j/3vCgsrxhMe8IwlC2t8oY78OsBPhwzyumg9sHHEW8241/9E8SFqXrhRCt1UazUmaYntUo65dVHe3ex/LygZ5xOM/Bu5zBYFzddfmHUbXU0UbMLOSW165haFSUz2SHrT9cUkPAJ8HXgem2l6jhO1GBK3lU+g+3fMucHvhiaNMm3pVQS4uRG4Rd6smUY2fn28A99u+Ij7h7WO7rDfNQCR1uDlQiPf/HrAoYS4Rwhd0DnC+7RPKnLcYYVRQcKu6mTBK/qCMfa4Ms5KeI4S3vhnbsxRBSOdV4OuOUVJ5fC6L6u8xeq80oq+lgyuyXxb4Lj3nNXfM2OxPCP/dhjB6LrAE4XG1rJ5s7EyHEUaf1QTOcxGnf6baXqdG+/8juHTtRFgMNXCB7XLz50RXNtEVLFNLMMY0YA2gllE3Cu6KPyaEMd9EWNQ6xnYPnYqMJ81qtk9RCMX+SClPmkR3Uoebkzj3eWGtK/TxC/lz28fWaJ87w6ykC4Brbd8c93chjKQvJuimfjyWb2p7oqSSsocuo/lQYkTfDjxaPKIvMUqsqYOTdAthYepYwijoKwTf1uMyNqsAqxLc2Y7PnP4u4emi2iN2zcRV9zXp3vmX9fOVdD1whHOKt8SFraHuHqlYyu4ThBX95wj3ciXgK1XaVPOoO9pPsT1K0l6ENYpvA3e5hOuZpN8RPWlsfyzer1ts9/CkUZ2uiwMW22nLuRE6mzz2D+SwnUL4Uk3OlD2Stz2Fcwg6A/W+zxMIHdo8givSu3F7HTi1gfdzYvH7BMZXsF+FoNgG4WljiTJ268T/Nym1lTnna4SIsTcJimKzCFMoldp/V7wv4+ha1BtbxnYooTP7C2Hl/hhCp1vx/hB+gAv7axXuWZXztgEOjq+XJfh2l7N9LP5/IbBrfP1wGdtJ8f/sZ7Sc7fKZv1mPrVGfof6ypUWz+pgkaTN3xZ9XY7KksYRV5flRai4dLlpPhtlXJB1HcGmCEIn1ahyJzvdwKDfKyLRnw6L9U4FTJZ3qMtMlpVD+DA6Fx+NXosvUy0CpeWmUSVFDyGiwIkFJrdSUwrejbSmBFFM6rf1RBJ/nB2zvEOfTKy0QAvxfleNZ/kDonM+K+wcQor2+WOGcwbafLOzYfipOM5Ql+6REeNIZTJCxLPekdIOkJwg/MN+M0zwlp7zI4Unj+l0XByRpSqEO4gdzDeB5QgdabX7s4hLFdolpCUnHEh5nP0l4fD4EuNz2WcW2mXNG0OWsD3Av8EOCW9DKjtFF5R4zMw0q97hZTuil5CNt9N/ciDAPeAlh1LSP7XIZHHYjzMuuROiIlgR+6BLhsqojRU0eJI23vVm8zscdHPWn2l6vynmrEObd/xnn7Nttv1vCbpp7TsX0KCs6fhGhQyvMpx4Y6y87rRXbvzFhNFq4T4+U+4zG48sQtB464ntY0vZ/StgdSPhR3wS4lOhJY7uHm5rKCBNBwyU++wVphFsfPSJqKuEK0oElbHNnmHVYIT6izOFnMnb1jjK+k3k9lNDhTaT0CBFyZHCI7bohvnyb4J9aiXpS1KDaEyq+KGkpglDPrZLeJPywVqq7eNS9AuVH3ZMkbeEoeyjp44TUTJX4JvAtuvQy7iZEJFYi15OSpIMyr7OHetwj25cpBMoUPGn2dBlPGi+ARshAJI1w6yS67Gwbd++2/XAF2xUJI7fC49zdhFDZF4vs2gnKUHnCP5G0FmHBaSTdO5SSHaJy6POWOX8lwhTBF8ocv5Ow0n0wsB1BJPzhcqNQBZ/PM4AtCSO5+wkr5NNL2J4GvEXw5zyCkKJmmu2yzvqqM6FiXKwaDtzkynrENY+6FRKKrk3wOwZYGXiSME9e8ilJmazDcb8dWMQV8qzlfVKSlC0fSuhMJ9nuEUKs4OZ4lu0pmbKTbZ9crj3RJvuducsN0GrodzR7Erk/boR5vscI/pGnEBZZjqhgfyuh8xkUtzHArWVsxxHCLPO052HCKGhzYNPCVsF+AmFKZDKhsz2YHItghFHNtArHP0KYP9027q9MEE4pZ/8A8OXM/fkSofMqZdtGiGK7Brg2vlaV9j5ezSbatQNP1PF5eDD+Pzn+X/DLLmVbcvGICotI8f4sntlfnOB2V61dnyRkzvgl8Mmc72kpwg9NqWMvxs/cQZmySVXqy/WdGahbGuHWQZyj3NL2e3F/GMGhu9wcbo9AgFJlsfx6wtzbrXRfYKukFTDR9qY52j/B9ujsnJ4qZ4g4i67H9jaCM/5ztkvlb8tNqblFSQ+7ghpWzvqvAY50XMCpYpvbxavOUfeH6e52VvZ6eT4/8VhdT0pFdQwmeC700IiQNIkw9fMnwkj9KIJXSaXgk1zfmYFKmsOtD9E9ZLUjlpXj9bhaf0Xc35/gWlWKv9ClMlXo5CrVDfA3Sf9DCAnOahG8Uca+qj5vEdk5xnnAFS4SxAZQHRkcIv+QdDzBy8KEBZkb4yIOtt+ox59T9SVUXBqYGm3f6zL1HmXaDt0TQx4K/N12ycSQCtk8fkWQTfwvYWT7OCHooxzFedZGUyHPmsOiV6ek4a7i45tpVzYYpo0g3VlOq0Gx3s9JOpngPje82iXI950ZkKQOtz4uBh6U9Ne4vychdUo5DiHMmZ5O+FDfR5hWmE9cYFrR9jlx/yGC76SBajoABR2E7OKWCRkjSvFlwpfqcIIf6EqEMNOS2L40uv5g+7UKdnVlcCAIpkBXPH7hi7gfXe9jpkIE3ueoYZEsMpYgan53Ufm2lE8fnnXxKugE7FfKsOhvdkFcPFsW2FTSW7avLXHajwghsf+0vbFCMshqTwpHA9dIejnuL0/4UarETEI26VqflLIC5POA591zjWENwnTRfO8R2ydL6iCkrq9E3u/MwKTZcxr9dSO4xBwZt43L2KxU4fzdivbvzdoTFniWIcx/jmtw24+qsUwEpbMZwBuEYIDXCJ4T5equeR6U4O/6kcz+Vwhf5jOBZYrbR1hMe46gnFXynhedcwOwQYnyDYC/VThvY8Lc53OE0VvJucZ6/mbAhPj/w0Bb4XW1+0Pwoz2ckJXi7OL7U+Lcr5TacnxG2oADG3E/M3ZVvzMDfWt6A/rTRphzOzp+4A8j5I2qZP8EMLJE+cHAv4rKxhftn515XTJSDfhu5vUXi479tEK7eixwkIkaypR9mzCXvGqmbDWCFsQxFeq/nuD/W+1+Tip0HARvhpcJIck/IoQqlzpnFcKIf3K8vycBa5WxrRSt9mjR/lqxricIouNHEEZ5ldpfz9/sn4RFr7MJU0xnUGYBrM77U/W+F9kvSYgoPJug9aHYsT8HXF/v/cyU5/rODPSt6Q3oTxsh3v9P8YNzHcE1qpL9Z4CnCA7xhbITCHN9KxbZPlOhnn+VKZ9U6nWp/Vi2P/A3wkh1bGa7nRIjstipjShRvmypDjpzvKZQVzIjO4KQy8mZ/aohyYSR6GSgo8zxpyuc+0zRfidwJ7BGpmx6levX8zdbjDB6bCeMOo+gzGi1nvtT9Jn4cw338HpCcMphBEnKO+J9GLUg9zNTnus7M9C3NIebj3UdfSujL2JFdSTbN0qaTVgU2pMQp785sJ17Sus9KOnrti/IFko6rMJ1VOZ1qX0Ic8evACPoHu76LlDKJ3KwS8ju2X5NlUNLaw11bZc0yEF4Zie6a6qW/GzGQIdPE+ZVdyJ0ECeXqX9CmXv6NULgRpbPxzpvV0i/fiXVF3Vq/puVWEgkU/+Jkv4FfN/2uMzx3PenqM215KhbLfOZvpDw+VjZpZXs8tzPArm+MwOd1OHmY74knoN6fdUTHFLZHEzoGO4jKCyV+jAfA1wn6QDCoyQEf9pFCAsMJasv87rUPg6RZs8TAgxqoayzf6VjLqM6VoIrCClkZhBW3e+G+Ysz3VbXY/Td/oSnhocIHeKhjm5GZTga+GsMRS10CKMJwR57FbX5OsL9HwbsEc/9sIIy1l9t31Ki/pr/Zq6wkBjduNYnJH1cP3Oo5vuTvVSZ1+XIfqY7JL1Y5vMJOe5nmfpr+s4MZJIfbg7iamzhCy66dHFLuj2pu1zhIoQPX0c5+3jOjnS5CE21fVsN7cm2pdC2obYHF9nnSslT9H6L7Re4/njOFoRV91vc5aO5FsHRf1LG7jZCDqw/l3g6qEj0BCh0ZBXvadF5SxNEZfZ1Zb3dmv9mVa53mO3zispquj8Z+0qfiWp/46qf6XhOzfeznvoHMqnDTSQSiT6ikrN7IpFIJBpI6nAXEEnVkucl+2Tf1Gv0d/uBROpwF5y8H55kn+z7+hr93X7AkDrcRCKR6CPSolmNLLVMu5dfsacX3VtvdLDUMu09yl96tLTe81xmM5hFeh4o4y4z1x8wWEN7Hijzdytbfxkq2aut5+/xHH/AkFLtAdzZM8tKxfYs1rOeufPeZ/CgxUrbv9/TW6mh73dwz7/vnM5ZDGlbtKS95/bMW1mxPWU8ouZ6NoNV4pwyX8281yhbf5lrVK6/5wXKfkah5Oe0XP0f8B5zPHuB/MY+tcMwv/5GR3VDYOIjs2+2veuCXC8vyQ+3RpZfcRAX/235mu3/b9UeCUwrosFDctl7biUX2cbQtmiZjq8Mne+X1cMuidapmLWmB548NZd9XgaNWC6X/bz/vJrLXoPyfd08L38i4t6+Rm9+Th/sFvNRH6+/0cFDN69ck2378k+PqHRc0q6E0Ot2QqbunxUdX5mQYmipaHO87Rsr1ZmmFBKJxIDBQGeN/yoRg1HOIUQ1rgvsL6k479wPgKsddID3o3rao3wdrqQOSVMkPSbpGoVEc3nO/14e+0QikciDMXPdUdNWhc0J+hDTHdIrXUmIQOx+uSD+A0EP+GWqkHeEO8v2KNvrE0I7v1HLSQq0AanDTSQSvUqOEe4ISRMyW9Z7YgXghcz+i7Esy8nAlyS9CNxI+USu81mQKYW7CXmxkPTtOOp9TNLRsWykpCcl/YGQy+j3wKJxhHxZPP5YoTJJx0b1eCRtJumRaPuLgp2kMZLOzpxzg6Tt4+tdJN0vaVIcfS8ey38maVqs75exbFlJf5Y0Pm6F5I6JRKIfY0yHa9uAGbZHZ7bzc15uf+AS2ysSND7+GAeWZalr0Syj2HSTpE0J+q4fJ6yRPqiQtfVNQtbQr7grJfQXHfMwSRpZ4RIXA1+3fb+kn1WwK7RnBGE+ZWfb70k6Dvi2pHMIohrr2LZC+msIE+Gn274nTnzfTMhgW1zvoUSfwY+s0NMTIZFItB6dNScEqchLhEwoBVaMZVm+CuwKEPuqoQQlvv+WqzTvCHdRhZTQEwjJ434PbENQU3rP9kxCPq5CKuTnC51trcROcQnb98eiy2s4bQvCxPa9sX1fIQhVvw18APxe0ufpEvLYGTg72o4FliyMiLPYPr/w61fK9SuRSLQWBjpwTVsVxgNrSlpVIf/ffmRSC0X+TZDNRNLHCGLrZVNQQf4R7iz3zB5ayb6SdN48unf4ZRz5ajpHhLTj+xefIGlzwk3Zm6Bkv2OsY4sKMnSJRKKf0ogRbpSSPJzw9NsOXGR7qqRTCGmSxgL/S8hldwyhrx/jKoENjXALuxvYU9JiClqie9EzaV+BueoSrn6VoDf6IUmLALsB2H4LeFfSx6NdNoHfc8AoSW2SViKsJAI8AGytoBOKpGGS1oqj1uHRN+4YoJB2+xYyE9ySSqabTiQS/QsDc+2atqp12TfaXsv26rZ/EstOjJ0ttqfZ3tr2RtGZoJRmcjcWOPDB9iRJl9Cl5H6h7cll5mjPBx6RNMn2gfHX4iHC3MgTGbuvEn45CmlPCmLL9wLPAtMIqaUnxTa8JmkMcEXsvCHM6b4LXB/nVkTI0QUhid05kh4h3IO7qNHjIpFItC6ubbqgaeTqcG33mOeM5b8Gfl1U9hzd1euxfRyZlN+2zyRkaC0ZEuxtAAAgAElEQVRmqu0NASQdT5gzJg7XDyzThtsIWU6L2byE7Qyqp5lOJBL9DUNH6/a3LRva+1lJJxDa9zwwprnNCdoIecJ1f/psvtRN31u1x+9C08kbqktbvoXFvKG67Wuvkct+7rIlxwdlmXfPlFz2eaknVLfVrtEXIeULQog0a11assO1fRUh22cikUjkQHRUzf3ZPFqyw00kEol6CItmrdvhNl28RtLMzOvPSHpK0irNbFMikeifBD9c1bQ1g5YZ4UraibCA9qmYzruWcwbZ7v2JsUQi0W/oTCPcykjaDrgA2M32v2LZSEm3RQ2EcTEEF0mXSDpX0oPAadHn9iJJD0maLGmPzPl3R22FSZK2iuXbS7pD0rWSnoi6Dq37F0okEjWTRrjVWQS4DtjedtYX9yzgUtuXSjqEMPrdMx5bEdjKdoeknwK32T4khgU/JOmfhHjmT9r+QNKawBXA6Hj+xsB6BDm1e4GtgXuKG5bVUhhKPjHuRCLR9xjR0RrjyJK0QsvmAvcRgh2ybEmXjsIfCZoNBa6x5wta7gIcH3UR7iCE+64MDCYETzwKXEPQWijwkO0XbXcCU4CRpRqW1VLIk8YlkUg0j06rpq0ZtMIItxPYBxgn6Xu2f1rDOVmNBgFfsP1k1iBKPb5KCOdtI4jYFJided1Ba9yHRCKxgBgxx60rNNUKI1xsvw98FjhQUmGkex9dOgoHUl6f4WbgiMI8rKSNY/lw4JU4iv0yQYAikUgMYELgQ1tNWzNomZGd7TcUkrbdJek1grjMxZK+Q5A8O7jMqT8CfkPQaGgjaC3sRsgv9GdJBwE3UVm5LJFIDBBS4EMFsvoMtl8AVs0c3rGE/Zii/VnAYSXsngY2zBQdF8vvIMz1FuwOr6vhiUSi5bBFh1viwb0kTe9w+wsa1E770h+q2T6vNsJBT75Q3SjDH9ZeqbpRlpw6BwD++PrVjTLowceqG2Xtc6bc7njymVz2bU9Wt8nS26nq24YNy2VfD7n1L2qQKexGXg/KvPU3gM40wk0kEoneJyyatW631rotSyQSiZwUFs1alZZtmaTlJF0uabqkiTEj714NqntmdatEItEf6bBq2qohaVeFzOPPRF3u4uOnK2QWnxI1YN6qVmdLjnCji9d1hEizA2LZKsDuRXZJSyGRSMynUZFmktqBc4BPAi8C4yWNtT1t/rXsYzL2RxAiWCvSqiPcHYE5ts8tFNh+3vZZksZIGivpNmAcgKTvSBofdRd+WDhH0peixsIUSefFm0jm+Ig4cv5sX72xRCLRu3S6raatCpsDz9iebnsOcCWwRwX7/QnyARVp1Q53PWK+sjJsAuxt+xOSdgHWJNygUcCmkraLaYv3BbaOmYY7yKTnkbQc8HfgRNt/76X3kUgk+pAgXtNW01aFFYCs69CLsawH8el7VeC2apW25JRCMZLOIWgpzCEM82+1/UY8vEvcJsf9xQkd8IbApoRHAYBFCYI2EHQWxgHfsn1nhet2ide05UvXkkgk+h4j5tYe2jtC0oTM/vm2z6/jsvsB12b0XcrSqh3uVOALhR3b35I0gphMkp5aCqfaPi9bQZxTudT2CSXqnwdMBD5FyApcknjzzwcYPnjZFk5Nl0gkILj95gh8mGF7dJljLwFZZ/cVY1kp9gO+VcsFW3VK4TZgqKRvZsrK6SPeDBwiaXEASStI+jBhBLt3fI2kZTKZJAwcAqwj6bhSlSYSif6I6Kxxq8J4YE1Jq0oaQuhUx/a4mrQOsDRwfy2ta8kRrm1L2hM4XdJ3CVoK7xHCcxctsr0lztfeH6cOZgJfsj1N0g+AW6LGwlzCr9Dz8bwOSfsDYyW9a/u3ffX+EolE72ByjXDL12PPk3Q4YUDXDlxke6qkU4AJtgud737AlXZtIXUt2eEC2H6FLrWwYi4psj0DOKNEHSWz/xb0G2zPJkwrJBKJAUKjBMht3wjcWFR2YtH+yXnqbNkOt+XoNJ71QXW7SNti+TJE/HHDNXLZt41aPZd955Rp1Y2KeG+lRasbZVh06Ea57Afd/Uguew3K+XFtz6cf0b78cvnqz6kr0PnKq/mqX3H5XPaQX4O087l8Gh55/wY1DvwCsxdcA8E0T1y8FlKHm0gkBgwhTXrrdmut27JEIpHITfMSRNZCU70UJHVkYpGnlIpXzlHXzPj/RyVdW8FupKR8OoKJRKJfYBoWadYrNHuEOytGgTUM2y8DezeyzkQi0X9II9ycSHpO0g8lTZL0aPR1Q9Kykm6VNFXShZKejwER2XPnj2AlrZfRUngkpksHaJd0QaznFkn5VocSiURLYqulR7jN7nAXLZpS2DdzbIbtTYDfAcfGspOA22yvB1xLSIdeiW8AZ8RR9GhCPDSE0N9zYj1vkYlqSyQS/ZewaNZe09YMWnlK4S/x/4nA5+PrbYC9AGzfJOnNKvXfD3xf0orAX2w/HYMjnrU9JVP/yFInd9NSUO+nR0kkEgtKa+c0a92Wwez4fwd1/jDYvpygoTsLuFFSISnl7IxZ2fptn297tO3RQzS0niYkEok+JCyaqaatGbRyh1uKe4F9AKIs49KVjCWtBky3fSZwPd2z+CYSiQFIg+QZe4Vmd7jFc7g/q2L/Q2CXuCj2ReA/wLsV7PcBHpM0BVgf+ENDWp1IJFqSQqRZq45wmzqHa5eeubY9MvN6ArB93H0b+FQUltgS2CzqIWT1EZ4jdK7Y/hlQ3Im/UTgebX7ZgLeSSCRahFZOItnsRbO8rAxcHdW/5gBf76sLe+gQvNbI2u0ffiLfBTqrahd3rz+nNsJTF5WT/SzPx34+I5d93rj8ji03yGXfds+U6kZZOvNJGHf+57/VjbL2H9SurQHQ/qFl8tWf834CtC01PJe95+VLCeic9xR35rBdcMlpG+Z2pg63Idh+mhoStSUSiYWTMKWQOtxEIpHoE1KkWQ4ymggjJR1Qg302smy0pDN7u42JRKI1aXW3sFYe4Y4EDgAur/WEuMA2oaphIpEYoLT2lELrtix4F2wb3cWOiSPZu6O+wiRJWxWfIGl7STfE15tLul/SZEn3SVo7lo+R9BdJN0l6WtJpffy+EolEL9KgnGa9QiuPcI8HjrW9G4CkxYBP2v4gitBcQdBHKMcTwLbRhWxn4Kd0aSaMIiy+zQaelHSW7fxLwolEoqUIXgqN0UmQtCshdVc7cGF0My222Qc4mTCb8bDtitOgrdzhFjMYOFvSKEI47lpV7IcDl8bO2fH8AuNsvw0gaRqwCtCjw+2mpTAkn7tNIpHoexqVYkdSO3AO8EmC6NV4SWNtT8vYrAmcAGxt+81ChvBKtPKUQjHHAK8CGxFGtkOq2P8IuN32+sDngKwYQm4thcGD8uUoSyQSzaFBUwqbA8/Ynm57DnAlsEeRzdcJqoNvAtiu6sjdyh3uu8ASmf3hwCu2O4EvUz1f3nDgpfh6TMNbl0gkWo4GeimsQPen3hdjWZa1gLUk3SvpgTgFUZFW7nAfATokPSzpGOC3wFckPQysA7xX5fzTgFMlTaZ/TZ0kEokFIIcA+QhJEzLboTkvNYigrb09sD9wgaSlqp3QUmQ0EeYCOxYdzqp9HRftnqNLO+EO4I74+n66z/P+IJZfAlySud5uDWt8IpFoKraYV7tb2Azb5RbeXwJWyuyvSNcTc4EXgQdjX/WspKcIHfD4chds5RFuIpFI5KZBUwrjgTUlrSppCLAfMLbI5jqisFZM9bUWML1SpS03wm1V9MEc9NRztdsPGVzdKIPn5VtZdUc+sZu1DskfD7La+EVy2T+714jqRlnufzSXedvwJXPZd87KJy6jJZaobpShrT2f+1HHm2/nq3/ROkTv58zNf04O8rap871qM3+NpTCHu8D1BHfSw4GbCetFF9meKukUYILtsfHYLtHTqQP4ju3XK9WbOtxEIjGgaFTYru0bgRuLyk7MvDbw7bjVRMtOKUjqiFFmU+PC2f9GWcZq532vBptLJKVU6onEAKPVBchbtsMlJpiMmXU/CXyakLW3GlU73EQiMXBp5dDeVu5w5xMdig8FDldgjKSzC8cl3RB1FH5GV9qey+KxgyQ9EkfJf8xUu13UWJieRruJxMDAhnmdbTVtzaDfzOHanh7D7cqGz9k+XtLhhdTrktYjuINtZXuGpKzk/vKEtOvrEFYfr+291icSib6iWdMFtdBvOtw62RG4xvYMANtvZI5dF6PWpklartTJ3bQUNKy325pIJBaQRmkp9Bb9psONKc87gP8C8+g+HVKH/0w3PYWSfyHb5wPnAwxvH7HgCZcSiUSv4xbucPvFHK6kZYFzgbOjK8ZzwChJbZJWIghNFJgrqeAEexvwRUkfivXky+KXSCT6Ha28aNbKI9xFJU0hyCrOA/4I/Doeuxd4FpgGPA5Mypx3PvCIpEm2D5T0E+BOSR3AZJKQTSIxYLHTHG5d2C4bxhNHuQeWOXYcUWch7l8KXFpkM6Zof/EFaWsikWgVREdKk55IJBJ9QyvP4aYOt0Zs43nzaraXcmoj5KgbgLacaUScT3sBYPpBK+ey3+GmSdWNMvxzw4pKdj3oeGdmLns6873nvPoXHa/n02poG5ZPxF7t+UdqefUjctefVxshz+c0/0e0B43SUugtUoebSCQGDg7zuK1K6052VEFS2eGOpPvqPTeRSPRvkpdCHyFpkO15tnukUE8kEgMft/iiWeu2rEaihsLdksYS3MTmj2AlLS/prqit8JikbTPn/STqKzxQLtIskUj0P+zatmbQ7zvcyCbAUbaLU6cfANwctRU2AqbE8mHAA7Y3Au4iZN9MJBIDAFs1bc1goEwpPGT72RLl44GLYuTZdbYLHe4c4Ib4eiJB/rEH3bQUSGnSE4lWJ4xeW9dLYaCMcEv6qti+C9iOkPztEkkHxUNzY/AEBGeUkj88ts+3Pdr26MGqR64hkUj0NUmAvElIWgV41fYFwIWEqYdEIjGAadQcrqRdJT0p6RlJx5c4PkbSa3GNaIqkr1Wrc6BMKZRje+A7kuYCM4GDKpsnEon+jBGdDfBSiNrb5xCmG18Exksaa3takelVtg+vtd5+2+EW9A9s3wHcUeZYDx2F7PH4+lqS+HgiMWBokAPC5sAztqcDSLoS2IPoCVUvA3pKIZFILGQ4l5fCCEkTMtuhmZpWAF7I7L8Yy4r5QkzhdW2Uiq1Ivx3h9jUdSy/GW5/ZuGb7pa6akKv+tsXyeUF0zpqVy37QR+pwNf5gTi7z23daPZf9M79cLZf9GseOz2WfV2/CH/SuDoE/mF3dKGvfln9hR+0533PuK+Qkp55FQ6j9Tc2wPXoBrvQ34ArbsyUdRnia3rHSCWmEm0gkBhQN8sN9CciOWFeMZZnr+HXbhV/RC4FNq1XalA5X0vclTY1D8SmSPt7g+pOWQiKxEGKgs1M1bVUYD6wpaVVJQ4D9CMlm5yNp+czu7oRkCBXp8ykFSVsCuwGbxKH4CGBII6+RtBQSiYUUAw3wsbU9T9LhwM1AO3CR7amSTgEm2B4LHClpd0JGmjeoIZtMM+ZwlyfMncwGKGTUlfQccDXwaWAWcIDtZyR9jpDqfAjwOnCg7VclnQysDKwW//+N7TNjXTNtLx5/ga4CliS812/avjva/ITQ8c8C9rD9al+8+UQi0bs0SifB9o3AjUVlJ2ZenwCckKfOZkwp3AKsJOkpSb+V9InMsbdtbwCcDfwmlt0DbGF7Y+BK4LsZ+3WATxFcOE7KJI8skLQUEomFDde4NYE+H+HanilpU2BbYAfgqkwUxxWZ/0+Pr1eMNssTRrlZzYS/x5HybEn/BZYjuG8UaJiWwpBhS9fzdhOJRJ/SPGGaWmjKopntDtt32D4JOBz4QuFQ1iz+fxYhPfoGwGFAVtQg62fTQxOhkVoKgxYZlus9JhKJJtHCI9w+73AlrS1pzUzRKOD5+HrfzP/3x9fD6XLH+ErOayUthURiYcLgTtW0NYNmLJotDpwlaSnC6t4zhMf23YClJT1CGLnuH+1PBq6R9CZwG7BqjmttT9JSSCQWMlp3SqEZc7gTgR5uWzHL7S9sH1dkfz1wfYl6Ti7aXz/zOmkpJBILKy2cRDKF9iYSiYFF6nCrY3tks9tQifY33mP4ZQ/Ubr9KVR2Lbsx7/oXqRgvAvP/0vpuxBuX7OK3x7f/msr/55SnVjTJ85pP7VjfK0DH1yVz2eXEf6Aq0cF/TNzQo8KG3aJkON5FIJBpBsxJE1kJFLwVJt0v6VFHZ0ZIullRx3lPSSEkHNKKRiUQiUTOdqm1rAtXcwq4giDZk2Q+42PbeVc4dSYj0SiQSiT5Drm1rBtU63GuBz0a1HCSNBD4KvCDpsVjWLukXksZH9a/D4rk/A7aNamDHxPw/f5F0k6SnJZ1WuIik30UB4KmSfpgpf07SqbGOCZI2kXSzpH9J+kbG7juZ6/8wlg2T9HdJD0t6TNK+sXxTSXdKmhjryir+JBKJ/kytQQ+tGNpr+w1JDxEEZa4njG6vpntzv0rQQNhM0iLAvZJuAY4HjrW9G4SEa4Qgh40JfrZPSjrL9gvA9+O12oFxkja0/Uis/9+2R0k6HbgE2JoQbfYYcK6kXYA1CXoKAsZK2g5YFnjZ9mfj9YfHEN+zCGI1r8VO+CfAIfXdvkQi0Vqo3y+aFaYVCh3uV4uO7wJsKKkwxTCc0AGWShcwzvbbAJKmAasQ0ljsE3ULBhHUxNYFCh1uQYPyUWBx2+8C70qaHYMndonb5Gi3eLz+3cCvJP0cuMH23ZLWB9YHbo1+v+3AK+XeeFZLYSj5MjIkEokm0cKLZrV0uNcDp0vaBFjM9sQ4tVBAwBG2b86eJGn7EnX10D6QtCpwLLCZ7TclXUJpvYTOovM7Y/sFnGr7vOKLxTZ/BvixpHHAX4Gptres+I4jts8HzgdYUsu08J8xkUjMp7PZDShPVS0F2zOB24GL6FLzynIz8M2CNKKktSQNA94FlqihDUsC7wFvS1qOMH2Rh5uBQyQtHq+/gqQPS/oo8L7tPwG/IOgoPAksG0XQkTRY0no5r5dIJFqVgh9uLVsTqNUP9wrC6LDYYwGCKMxIYJLCc/prwJ6EKYEOSQ8T5l7fLFWx7YclTQaeIEwv3Juj/di+RdLHgPvjNMFM4EvAGsAvJHUCcwni43Pi1MeZkoYT3v9vgKl5rplIJFqXZnkg1EJNHa7t68goQth+jjAXiu1O4HtxK6Y4g+UlmTp2y7weU+a6IzOvLyk6P3vsDOCMotP/RRj9Ftc5hSDZmEgkBiIN6nAl7UroV9qBC23/rIzdFwgeXZvZrpiuO2XtTSQSiSKix9Q5hCnOdYH9Ja1bwm4J4CjgwVrqTaG9vcTMDfO59w7tZS2FvsAdvasVsOseX85lv90VD+Wyv3PDRXPZJ2pAOeZKGzUybUw9mwPP2J4OIOlKYA9gWpHdj4CfA9+ppdI0wk0kEgMH06jQ3hUIa0oFXoxl84leUCvZ/nutzWtGxoePSLoyRotNlHSjpEMl3VDG/sJSQ/lEIpEoSe2RZiNiBGthO7TWS0hqA34N/G+epvXplEL0YvgrcKnt/WLZRsDu5c6x/bU+al4ikRgA5JhSmGF7dJljLwFZjdUV6Ur1BcHldX3gjugd9RFClOvulRbO+nqEuwMhgeO5hQLbDxOiwhaXdK2kJyRdFjtnJN0haXR8PVPST6I+wgPRbxdJy0r6c9RTGC9p61j+iajDMEXS5DjBXVJ7IZFIDBAao6UwHlhT0qoKWjL70RX1iu23bY+wPTJ6TD0AVOxsoe873PUJaclLsTFwNGFFcDWCZkIxw4AHbG8E3AV8PZafAZxuezNCBuALY/mxwLdsjyKkZZ9VpL0wCtg0ai8kEomBQAM6XNvzCBnFbwYeB662PVXSKZLKPpFXo5W8FB6y/SKApCmEYIp7imzmAIW53onAJ+PrnYF11bUiumSMPLsX+LWky4C/2H4xdriltBfuKm5Q0lJIJPoXjZRetH0jcGNR2YllbLevpc6+7nCnAuV0dHvoLJSwmWvP13PP2rQBW9j+oMj+Z5L+TtBTuFdBTL2s9kIxSUshkeiHNElcvBb6ekrhNmCR7GqgpA0Jj/sLwi3AEZk6R8X/V7f9qO2fE+Zk1qGM9sICXj+RSLQI/VmAvKHE0elewM7RLWwqcCrwnwWs+khgdFwEmwYUxMmPVhAff4Sgp/AP27cAlxO0Fx4lhOTVIrKTSCT6A/1VgLw3sP0ysE+JQxdkbA7PvN4+83rxzOtrCZ0ltmcAPVK02j6iuCyWl9JeSCQS/Z0mjl5roZUWzRKJRGLBSR3uAEBCiyxSs/nQv+WL489TN4Bnz65ulK1/8JBc9vWgIYNz2Xe+916++qdNz2V/12ZL5bK/5N/jctmPWXmbXPaDPrJcLvuO10sqmlbG+dS3PW9e/mvkukDf937qzwLkiUQikWgMLdnhStpTkiWtU8e5M8uUnyJp5wVvXSKRaGlaeNGsJTtcYH9C0MP+xQck1TUNYvtE2/9c0IYlEokWpkaXsIXCLawWon/sNoTswAWBm+0l3S1pLFGPUtJ1UW1sarHKj6TTY/k4ScvGsktieh0kbSbpvqjJ8FBBYyGRSAwA0gg3F3sAN9l+Cnhd0qaxfBPgKNtrxf1DbG8KjAaOlPShWD4MmGB7PeBO4KRs5VGI4qpY10aEsOBZvfqOEolE35E63FzsD1wZX19J17TCQ7afzdgdGRNUPkCQUVszlncSOlSAPxFGy1nWBl6xPR7A9jtRqKIHUad3gqQJc3tEDScSiVZDBC+FWrZm0FJuYZKWISSe3ECSCcnbDPydkEq9YLc9YWS6pe33Jd0BDC1Tbd2/Zd20FNo+1MLefYlEAmj5wIdWG+HuDfzR9ipRZ3Il4Fl6ai0MB96Mne06wBaZY210CeQcQE/FsSeB5SVtBiEJXL0LcYlEogVJUwo1sz8hI0SWP9PTW+EmYJCkx4GfEaYVCrwHbC7pMcJo+ZTsibbnEMKAz4pTErdSfnScSCT6Gy3c4bbUyM72DiXKzgTOLCqbTUhfXKqOxcuUj8m8Hk/3UXEikRggtPKUQkt1uIlEIrHApA63/yMJDar9dqm9PVf9nbPyeaa1LZYvA0XnB/m0FwDe2W+zXPbL3PnvfBfIGWefO+6/M1/9h+xec9JWAGYcNjyX/YcvnVzdKIOG5tPXAGhbKl+b5r3wYi779g8tk8u+440cehCN6CidtBQSiUSi72jQHK6kXSU9KekZSceXOP4NSY/GJLX3SFq3Wp193uFKWk7S5ZKmx0ix+yXt1dftSCQSA5NGhPZKagfOIawVrQvsX6JDvdz2BjFJ7WnAr6u1rU873Jj6/DrgLturxUix/Qg532s5P02BJBKJyjRmhLs58Izt6dGz6UpCFGzXZex3MrvDaqm1r0e4OwJzbJ9bKLD9vO2zJLVL+oWk8TFVzmHQU0dB0khJT0RthKckXSZpZ0n3Snpa0ubxvM3j6Hly1E1YO5aPkfQXSTdF+9P6+B4kEoneotbOtnqHuwLwQmb/xVjWDUnfkvQvwgj3yGqV9nWHux4wqcyxrwJv294M2Az4uqRV47FiHYU1gF8RkkKuQwhw2AY4FvhetHkC2Nb2xsCJwE8z1xpF8MXdANhX0koNeG+JRKLJiFxTCiMKoftxy7dqCtg+x/bqwHHAD6rZN/URXdI5hI5yDvA8sGFB0YsQTbZmPFaso/Cs7UdjHVOBcbYdk0KOzJx/qaQ1Cb9n2XQE42y/Hc+fBqxC91+zQvsOBQ4FGKphC/6GE4lEr5PDD3eG7dFljr1E0GgpsGIsK8eVwO+qXbCvR7hTCaNVAGx/C9gJWJbw43SE7VFxWzVm2IWMjkIk6+PUmdnvpOtH5EfA7bbXBz5H92iy7PkdlPnhsX2+7dG2Rw9RCkZLJPoFjZlSGA+sKWnVqDC4HzA2axAHcwU+CzxdrdK+7nBvA4ZK+mamrOBQejPwTUmDASStJS3QsHI4Xb9IYxagnkQi0Z9oQIcbFQQPJ/RLjwNX254aM8fsHs0Oj7rbU4BvA1+p1rQ+nVKIj/17AqdL+i7wGmH0ehxwDWE6YFL0ZngN2HMBLncaYUrhBwS1sUQiMdBpoFqY7RuBG4vKTsy8PipvnX0+h2v7FWImhxJ8j65FrwJ3xK1w/nPA+pn9MaWO2b4fKCyyQZzQtn0JcEnmnN1yvYFEItHapNDeRCKR6BtaObQ3dbi1YkNHRw7z3v2Z7Xz//Vz2bUvkT9s2/OoJuew7F8kX+59XPyKv9gJSLvP2/7yey37EedNy2Y99aWIu+91W2LS6URGd7+X7XNCWT/Oj4/U38tXfBJJaWCKRSPQFTdS6rYXU4SYSiYHFwtrhSppZThA8kUgkGk0h0qxVSSPcRCIxoFBOHeS+pNcDHyQtLmmcpElRO3KPWF4QoblM0uOSrpW0WDx2YhSxeUzS+dEvF0l3SPq5pIeicM22sbyc8M3yku6KepWPZex3icI2kyRdIymNwhOJgUDjxGt6hb6INPsA2Mv2JsAOwK8KHSiwNvBb2x8D3gH+J5afbXuzGJa7KJD1lR1ke3PgaOCkWFZO+OYA4OaoV7kRMEXSCIJP7s6xTRMIUSI9kHRoQdhiDvkzJiQSib6nEXq4vUVfTCkI+Kmk7QhaBysAy8VjL9i+N77+E0He7JfADjESbTFgGYIGw9+i3V/i/xPpEqrZhdLCN+OBi2K48HW2p0j6BEFQ+N7Y7w8B7i/VcNvnA+cDDG/7UOs+pyQSiS5a+JvaFx3ugQRxmk1tz5X0HF1CMsW3xpKGAr8FRtt+QdLJlBaeyYrOFIRvbi6+eOzoPwtcIunXwJvArbaLU68nEokBQCsvmvXFlMJw4L+xs92BIIVYYGVJW8bXBwD30NW5zohzq3tTnZLCN5JWAV61fQFwIUGp7AFga0lrRNthktYqV3EikehntPAcbq+NcBXS4cwGLgP+FrVqJxCEwQs8CXxL0kqi3jMAAA+cSURBVEXANOB3tt+XdAHwGPAfwrRANS6ktPDN9sB3JM0FZgIH2X5N0hjgCkmF0KgfAE8twNtNJBKtgBfe0N71gH/ZngFsWXxQ0khgnu0vFR+z/QNKqKfb3j7zegZxDtd2J6WFby6NW3E9txEW1xKJxABiofTDlfQNwgLY0b1RfzOwTecHH9R+Qs44/ry0j/hQLvvOt9/NfY2OLTfIZa95+YYWuv/hXPZ5ac+pH+E8f1+gfcklc9nn1UZ4+a9Vs273YMX9p+eyz/WZ7i/0so7JgtArHW5MEnluFZvnyMgsJhKJRCNY6Ea4iUQi0RRaXLymr1PsIOn7MS3FIzEC7OM5zx8l6TOZ/e0lbZXZ/4akgyqcf7KkY+trfSKRaHXUWdvWDPq0w40uYLsBm9jeENiZEtlyqzAK+Exmf3tgfodr+1zbf1jApiYSiX5KozpcSbtKelLSM5KOL3H825KmxcHjuOiGWpG+HuEuT0hNPBuCp4HtlyVtJuk+SQ9HnYQlJA2VdHHUX5gsaYeYPfMUYN84Oj4O+AZwTNzfNjuClXRk5oZcmWnHulGXYbqkI/v4HiQSid7ChEWzWrYKSGoHzgE+TYhM3V9S8SrmZEKA1obAtYQ8ihXp6zncW4ATJT0F/BO4ihBWexWwr+3xkpYEZgFHEfJObiBpnXjuWsCJhDd5OICkRYGZtn8Z93fKXO94YFXbsyUtlSlfh6DrsATwpKTf2Z5b3FhJhwKHAgydn1w4kUi0Mg1aNNsceMb2dIA4YNuDEC8AgO3bM/YPAD1cXIvp0xGu7ZnApoRO7DVCR3sY8Irt8dHmnZiieBuCvgK2nwCep3tSyFp4BLhM0peAeZnyv9ueHX15/0uXtkNxe8+3Pdr26MHkSx+TSCSaRO2RZiMK4lRxOzRTywp0n+58MZaV46vAP6o1rRlZezuImXhj9Nm3evFynwW2Az4HfF9SwbE0K/2V1WRIJBL9mJyBDzNsj17ga4YB3WjgE9Vs+3rRbG1Ja2aKRgGPA8tL2izaLBHDgu8mCN8QtQ5WJoQCv0uYCihQvF+4VhuwUhz2H0fQdEi6t4nEQMZGnbVtVXgJWCmzv2Is64aknYHvA7sX1qYq0deLZosDlxYWsgiT0ScC+wJnSXoYuJUgYPNboC2Ogq8CxsQ3dDth0WuKpH0Jso17FRbNMtdqB/4Uz58MnGn7rT56n4lEolk0RrxmPLCmpFXjYv1+wNisgaSNgfMIne1/a2lanz5K255IxoUrwwxgixLlB5eo4w166iBsmHl9d+b1NiXOP7loP0W7JRIDiEYsmtmeJ+lwghJhO3CR7amSTgEm2B4L/IIwiLwmamv/2/bulepNc5eJRGLgYKBBOc1s3wjcWFR2Yub1znnrTB1ujaitjbZFa3cN85w5+eofMiSXfee7M3PZe26+9gCoI184TufQ9lz2g4cOrW6UwXnb8/77uezblx2Rr/538gkCaXC+v/FKB/eYMqzKE7/cKJf9moc/mO8Cbfn+xnR25LNvBC0c2ps63EQiMaBoZfGaPtdSKIWkjrjoVdhG9uK1tpd0Q2/Vn0gkmkuDvBR6hVYZ4c6KmXVLImlQDIZIJBKJ8iS1sPqQNEbSWEm3AeNi2XckjY/aCD+MZSMlPS7pgqhCdksM90XSGpL+GTUaJklaPVa/uKRrJT0h6bKYlieRSPRzQuCDa9qaQat0uItmphP+minfBNjb9ick7UJIfb45IWBiU4WMvMTyc2yvB7wFfCGWXxbLNyK4o70SyzcmZKNYF1gN2LoX31sikehLOmvcmkCrTyncGv1uAXaJ2+S4vziho/038KztKbF8IjBS0hLACrb/CmD7A4A4mH3I9otxfwohN9o9xRfvJl6jYQv4FhOJRF/QrNFrLbRKh1uO9zKvBZxq+7ysQVxgK9ZGWLRKvTVpKdg+HzgfYHj7iNb9KyYSiUCaw20YNwOHSFocQNIKkj5cztj2u8CLkvaM9otIShqLicSApmFaCr1Cq49w52P7FkkfA+6P0wIzCfqTlTyrvwycF8Px5gJf7PWGJhKJ5pKmFCpju4eKl+1LgEuKys4AzihRxfoZm19mXj8N7FhkO50gD1mwObyOJicSiVbEzctXVgst0eEmEolEw0gj3P6POzvp/KCq3GX2hHz156kb8seo542BB8jpnjz4tXzaBZ2zc77nXsbv5Wu/5+WLxdHQfFlDOt+blcseYM0jHsplP++fK+eyH7LH67nsnedv3KjQptbtb1OHm0gkBhbqbN05hT73UsjoJjwm6W9FyR0XtO7Rks5sVH2JRKKfYVo68KEZbmGzbI+Kwt9v0MCcZrYn2E5pzxOJhRRRW1jvwhraez8xE2axipeksyWNia9/VkjLI6mQDv2LcZT8sKS7iuuQtLmk+yVNlnSfpLVj+RhJf5F0k6SnJVXNJZ9IJPoRdm1bE2hahyupHdiJojxBJew+BOwFrGd7Q+DH8dCJwKeiTkKptBZPANva3vj/27v/WK/qOo7jzxcXLkIritAsMaDENTSxebO2lquVS5eMVEpYbdmaZIu1sma2iKi2sh/DarmV5o9+jGEyWMzwx8RcrJRfmghuFgIKMiOU0PzBj3tf/XE+3zh8Offec67f+/0e6P3Yvrvf7+e8z+d+7h37cO7nx/uTYr+Xu3Y22Tlq7wQuk3Rqwf0hhGNRizpcSRdIelzSFknXFFw/LyXFOiRpVpmmdaLDHZPyFzwDvIns0MiB7ANeAW6SdAnQmEr+C3CrpCvIzhxqNo7srKFNwHXAGblrq2zvS/kVHgMmFX1jSXMbZ9YfpF4z6iGEAi0aw00PhNcDF5IluZojaVpT2FPA5cDiss3r2BguWScnDo/hHmpqzwmQHeZGliFsKXARcFcqvxKYT3aU8Yb0JJz3XeBPaax4RqO+pHQuBds9tntGUW1JTwihM9TXV+o1iHOBLba32j4ALAFm5gNsb7e9kQpTcB0bUrD9EvBF4CuSRgJPkh1/PjqtXPgQQMqdMC4d6PZlYHoqf7vtNelQt39x5BnykD3hNg6Funy4f54QQh2UHE4YfEjhFGBH7vPOVPaqdHTSzPbDwEZgju0dwO+BTelrIw3ja4E7JG0kS6F4VSr/kaRH05DBX4FHmqr/IfB9SQ8T641D+P9gqnS4ExpDhuk1d7ib1/aOqDlvgu0ZufdXA1cX3HZuQT2XFMTdn17YfgA4PXdtfiq/lVyOBtsXlWx6COFYUH6N7R7bPf1ce5oj/2qeyOG/mIes08vCQgihpVq0DncdMFXSFEndwGwGWVFVRvypXZJGjqRr/PjS8b3PPjd4UL7+roq5DirGu7di7gXAFU960z+r7bPven21TYZ9L1XLdVD5qLruUZXCR4yuOJE63Pk1qP47HVExN8KOedMrxb912TOlY/Vkd6W6+9WCNba2D0maR5aHuwu42fbmlOp1ve0Vkt4NLAfeAMyQ9O10zFe/osMNIRw/bOhtzb7dNFG/sqlsQe79OrKhhtJqO6Qg6WRJSyQ9IWmDpJWSTh/8zkHrXSjpq61oYwihhmq806yWT7jp2PLlwK9tz05l08k2Svy9k20LIdRcjfPh1vUJ94PAQdu/aBTYfgQ4P3ec+tOSbgGQ9ClJa1P5L9MukcbWvIdSvoVVufqnSbpf0lZJkewmhOOFgT6Xe3VAXTvcM8mOOz+C7QVpl9oHyDKN/Tydc3YZ8L50rRf4pKQTgRuBS1O+hfx5Zu8APkK23OxbkqrNloQQasrZ5GSZVwfUckhhIGm44XfAItsb0kziOcC6NCs9BtgNvBf4s+1tALbzywb+aHs/sF/SbrKhip0F32suMBfghBFHHbsWQqgb07JJs+FQ1w53M9Bf9p2FwE7bt6TPIhvr/Xo+SNKM5htzSudSAG4AGDfqpPoODIUQDosx3MruA0bnt9pJOkvSN4EPk+VgaFgFzJJ0UoobL2kS8CBwnqQpjfK2tT6E0DmxSqEa25Z0MfATSV8jS8+4HRhLlkBibRo+WGF7gaT5wD2SRgAHgS/YfjB12MtS+W7g/A78OCGEtulcZ1pGLTtcANu7gE+UjL0NuK2g/E7gzqayhU2fzxx6K0MItWKgxodI1rbDDSGEIYkn3GOfDx2id8+e0vHqrrYv3Pvrd6JE9xPl98FD9nBRRe++56vd0FcxH8TIav+8deBgpfje56u1f+SbT64UPxS9e/dWih8xdmyl+EmLn6oU33PHttKxG2e/UqnuYq3b2jscosMNIRw/DO7QGtsy6rpKAUm9aefYJkm3Sxrwv2JJ/0lf3yJp6QBxk1PS8hDC8Sh2mg3Jy7bPTpNaB4Ary9xke5ftUidohhCOQzVeFlbnDjdvNXAagKSr0lPvJklfag7MP8FKOiOXY2GjpKkprEvSjZI2S7pH0pj2/SghhGFjZ6sUyrw6oPYdbjpg8kLgUUnnAJ8B3kO2dfcKSe8a4PYrgZ+mHAs9HN6+OxW4PiUL/jdw6XC1P4TQZjV+wq3zpNkYSX9L71cDNwGfB5bbfhFA0jLg/Rw+cLLZA8A3JE0Eltn+R9owsc12o+4NwOSim4/IpUC12dwQQid4SKebtEudO9yX05Pp/1Q9MsX2YklrgI8CKyV9DtjK0bkUCocU8rkUXqfx9V3cF0LINNIz1lTthxSarAY+JmmspNcAF6eyQpLeBmy1/TPgD8BZ7WlmCKFjWpSeMeXTflzSFknXFFwfLem2dH2NpMmD1XlMdbi2HyI74nwtsAb4le3+hhMg2xq8KQ1NnAn8ZtgbGULoGAPuc6nXQNIhBteTzR9NA+ZImtYU9llgr+3TgOuAHwzWvtoOKdguTEBrexGwqL9429vJOldsXwtc2xT6XON6ivlxa1ocQug4u1XJxc8FttjeCiBpCTATeCwXM5MsXSzAUrIDEWT3PyNX2w43hBCGokWTZqcAO3Kfd5KtjiqMSceq7wPeCPSbAyA63JJeYO+ee/tuf7Lg0gSKfsH9bwsvju9f5+J31aw9VeOLUyP0H79vmNtT/Pts7feoGv/i8MbfVTxr0l/8pMLoCl5g7933eumEkuEnSFqf+3xDmigfNtHhlmT7xKJySett95StJ+Ijvu5tqlt8FbYvaFFVTwOn5j5PTGVFMTvTfoFxwLMDVXpMTZqFEEKbrAOmSpoiqRuYDaxoilkBfDq9nwXcN9D4LcQTbgghHCWNyc4D7ga6gJttb5b0HWC97RVkm7F+K2kL2WT87MHqjQ731as65hPxEd/u73Gsx3eE7ZXAyqayBbn3rwAfr1KnBnkCDiGE0CIxhhtCCG0SHW4IIbRJdLghhNAm0eGGEEKbRIcbQghtEh1uCCG0SXS4IYTQJv8Fvgx1Vd28+KgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 混淆矩阵\n",
    "confusion = torch.zeros(n_categories, n_categories)\n",
    "n_confusion = 10000\n",
    "\n",
    "\n",
    "def evaluate(line_tensor):\n",
    "    hidden = rnn.init_hidden()\n",
    "    \n",
    "    for i in range(line_tensor.size()[0]):\n",
    "        output, hidden = rnn(line_tensor[i], hidden)\n",
    "    \n",
    "    return output\n",
    "\n",
    "# 最好是有一个测试数据集，我们这里随机从训练数据里采样\n",
    "for i in range(n_confusion):\n",
    "    category, line, category_tensor, line_tensor = random_training_pair()\n",
    "    output = evaluate(line_tensor)\n",
    "    guess, guess_i = category_from_output(output)\n",
    "    category_i = all_categories.index(category)\n",
    "    confusion[category_i][guess_i] += 1\n",
    "\n",
    "# 归一化\n",
    "for i in range(n_categories):\n",
    "    confusion[i] = confusion[i] / confusion[i].sum()\n",
    "\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "cax = ax.matshow(confusion.numpy())\n",
    "fig.colorbar(cax)\n",
    "\n",
    "# 设置x轴的文字往上走\n",
    "ax.set_xticklabels([''] + all_categories, rotation=90)\n",
    "ax.set_yticklabels([''] + all_categories)\n",
    "\n",
    "ax.xaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "ax.yaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "演示越深值越大，因此可以看出中国人名最容易和韩国人混淆。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试\n",
    "\n",
    "我们首先实现predict函数，它会预测输入名字概率最大的3个国家。然后手动输入几个训练数据里不存在的人名试一试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "> Dovesky\n",
      "(-0.88) Russian\n",
      "(-1.25) Czech\n",
      "(-2.29) Polish\n",
      "\n",
      "> Jackson\n",
      "(-0.70) Scottish\n",
      "(-1.32) English\n",
      "(-2.09) Russian\n",
      "\n",
      "> Satoshi\n",
      "(-0.87) Japanese\n",
      "(-1.57) Arabic\n",
      "(-1.99) Italian\n"
     ]
    }
   ],
   "source": [
    "def predict(input_line, n_predictions=3):\n",
    "    print('\\n> %s' % input_line)\n",
    "    output = evaluate(Variable(line_to_tensor(input_line)))\n",
    "\n",
    "    topv, topi = output.data.topk(n_predictions, 1, True)\n",
    "    predictions = []\n",
    "\n",
    "    for i in range(n_predictions):\n",
    "        value = topv[0][i]\n",
    "        category_index = topi[0][i]\n",
    "        print('(%.2f) %s' % (value, all_categories[category_index]))\n",
    "        predictions.append([value, all_categories[category_index]])\n",
    "\n",
    "predict('Dovesky')\n",
    "predict('Jackson')\n",
    "predict('Satoshi')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py3.6-env",
   "language": "python",
   "name": "py3.6-env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
